Overview
Imagine you have 5000 audio files in your Amazon S3 bucket and you want to move it to a new AWS Account. But there is one caveat you cannot zip the audio folder download it to your local machine and upload it to the new AWS Account.
So what you would do? Don’t worry you are in the right place for the solution
In this post, you will learn how to migrate the data/objects of S3 Bucket from one AWS account to another AWS account
Objectives
What is Amazon S3
Getting the Destination AWS Account Number
Attaching the policy in Source S3 Bucket
Attaching the policy in Destination AWS Account IAM User
Installing the AWS CLI in the Local Machine
Configuring the Access Key ID and Secret ID
Syncing S3 objects to Destination AWS Account
Prerequisites
1. What is Amazon S3
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. It can be used by all sizes of industries which can store and protect data for a range of use cases such as websites, mobile apps, backup, and restore enterprise applications.
2. Getting the Destination AWS Account Number
Once the Support center is clicked you can see a screen like below
On the left-hand side, you can see the Account number for your destination AWS Account Copy that number.
3. Attaching the policy in Source S3 Bucket
Now go to your source AWS account and then select S3 Bucket
Select the Bucket which you want to migrate
Click on the permission tab and select Bucket policy like below
In the policy editor add the below configuration
If your bucket has public access then do like below
4. Attaching the policy in Destination AWS Account IAM User
Goto your Destination AWS account and select IAM user like below
Click on the IAM option which will navigate to the screen like below
Click on the Policies on the left-hand side which would open a screen like below
Click on the Create policy and then click on the JSON tab
Now replace the policy with the below code
Once the policy is added click on the Review policy like below
Which will open a new page where you can fill in the policy name like below
Now click on the Create Policy button. Once the policy is created click on the User on the left-hand side like below
Now click Add User and then you can see a screen like below
Fill in the user name select the Access type as Programmatic access and then click Next Permissions
Now you can see a page like below and select Attach existing policies directly
Attach existing policies directly
In the Filter, policies type the policy name which we created earlier below
Click Next Tags and then in the last section you can see the credentials below
Download the credentials to your local machine because it will be shown only once
5. Installing the AWS CLI in the Local Machine
Now we will install the AWS CLI in the Local Machine for that open your terminal and type the below command (Mac and Linux Machine)
sudo pip install awscli
Once it is installed check whether AWS CLI is installed properly with the command
aws help
Now you could see an output like below
6. Configuring the Access Key ID and Secret ID
Now we will configure the AWS Destination account Access key and Secret key in the AWS CLI
Open the terminal and give the command
aws configure
which would ask for the Access key and secret ID like below
Configuring Access Key ID and Access Key
Add the Access Key ID, Secret Access Key and enter the default region of your Destination AWS account and then type table to show the output in table format else press Enter which will use the Default JSON option
Once the configuration is done it’s time for migration
7. Syncing S3 objects to Destination AWS Account
To Sync the S3 objects from Source to Destination AWS account use the below command in your terminal
aws s3 sync s3://YOUR-SOURCE-BUCKET-NAME-HERE s3://YOUR-DESTINATION-BUCKET-NAME-HERE --source-region SOURCE-REGION-NAME --region DESTINATION-REGION-NAME
Replace the source bucket name, destination bucket name, and source, and destination region like below
aws s3 sync s3://my-us-west-2-bucket s3://my-ap-south-1-bucket --source-region us-west-2 --region ap-south-1
Now click enter which will start copying the files from your Source Bucket to Destination Bucket :)
I have some doubts
Question #1
Q: I have multiple folders in my Source S3 Bucket and i want to copy only one folder to the Destination S3 Bucket for that what shall i do
A: It is very simple just change the command like below and click Enter which will copy only one folder
aws s3 sync s3://my-us-west-2-bucket/folder1 s3://my-ap-south-1-bucket/folder1 --source-region us-west-2 --region ap-south-1
Question #2
Q: I have added new audio in my Source S3 Bucket and i want to copy that one to the Destination S3 Bucket. If i give the above command will it copy all the 5001 audio files or just the 1 audio file
A: It will copy/sync only the 1 audio file and not the other audio files from your Source S3 Bucket
Question #3
Q: I have renamed a file in Source S3 Bucket and now i am giving the above command will it replace the existing file in the Destination S3 Bucket?
A: No it will create as a new file in Destination S3 Bucket keeping the old one as it is.
Congratulations!
You have learned how to Copy/Sync files from one AWS Account to another AWS Account. Happy Learning!!!!
