Multi tool use

Clash Royale CLAN TAG

Go - How to keep files in memory - transferring files between servers without storing them locally

I am in the process of writing a simple deployment tool which needs to take tar files from s3, extract them and then upload them to our staging server. I would like to do this without storing the files locally, keeping them in memory.

Here is my code to download files from s3

func s3downloadFile(downloader *s3manager.Downloader, item string) {
localitem := strings.Replace(item, s3base, "", -1)
os.MkdirAll(path.Dir(localitem), os.ModePerm)
file, err := os.Create(localitem)
if err != nil {
exitErrorf("Unable to open file %q, %v", err)
}

defer file.Close()

numBytes, err := downloader.Download(file,
&s3.GetObjectInput{
Bucket: aws.String(s3bucket),
Key: aws.String(item),
})
if err != nil {
exitErrorf("Unable to download item %q, %v", item, err)
}

fmt.Println("Downloaded", file.Name(), numBytes, "bytes")
}


I would like to avoid having to create the directories and files in this example and just keep everything in memory. I left out the step that extracts the files from my code. The next step would be to upload the files with go-scp like so:

// Finaly, copy the file over
// Usage: CopyFile(fileReader, remotePath, permission)

client.CopyFile(f, "/path/to/remote/file", "0655")


My question would then focus on the file, err := os.Create(localitem) part of this code, how can I keep a filereader in memory instead of storing the file locally.

file, err := os.Create(localitem)

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.