amazon s3 - Is it possible to copy all files from one S3 bucket to another with s3cmd?

Question

I'm pretty happy with s3cmd, but there is one issue: How to copy all files from one S3 bucket to another? Is it even possible?

EDIT: I've found a way to copy files between buckets using Python with boto:

from boto.s3.connection import S3Connection

def copyBucket(srcBucketName, dstBucketName, maxKeys = 100):
  conn = S3Connection(awsAccessKey, awsSecretKey)

  srcBucket = conn.get_bucket(srcBucketName);
  dstBucket = conn.get_bucket(dstBucketName);

  resultMarker = ''
  while True:
    keys = srcBucket.get_all_keys(max_keys = maxKeys, marker = resultMarker)

    for k in keys:
      print 'Copying ' + k.key + ' from ' + srcBucketName + ' to ' + dstBucketName

      t0 = time.clock()
      dstBucket.copy_key(k.key, srcBucketName, k.key)
      print time.clock() - t0, ' seconds'

    if len(keys) < maxKeys:
      print 'Done'
      break

    resultMarker = keys[maxKeys - 1].key

Syncing is almost as straight forward as copying. There are fields for ETag, size, and last-modified available for keys.

Maybe this helps others as well.

Answer 1

s3cmd sync s3://from/this/bucket/ s3://to/this/bucket/

For available options, please use: $s3cmd --help