✨Crafting Code with a Smile for 8 Years:) Merging the Formal Dance of Angular, the Playful Rhythms of React, and the Next-level Moves of Next.js 🚀
Overview
In this post, you will learn how to import CSV data to a Firestore database using GCP.
Objectives
What is GCP
What is Firestore
GCP Setup
Set up Firestore in GCP
Write database import code
Upload a CSV file to GCP
Import the data to Firestore
View the stored data in Firestore
1. What is GCP?
Google Cloud Platform (GCP) is a suite of public cloud computing services provided by Google. The platform includes a range of hosted services for compute, storage, and applications that run on Google Hardware.
Google Cloud Platform services can be accessed by software developers, cloud administrators, and other enterprise IT professionals over the public internet or through a dedicated network connection.
2. What is Firestore?
Cloud Firestore is a fast, fully managed, serverless, cloud-native NoSQL document database that simplifies storing, syncing, and querying data for your mobile, web, and IoT apps on a global scale.
Its client libraries provide live synchronization and offline support, while its security features and integrations with Firebase and Google Cloud Platform (GCP) accelerate building truly serverless apps.
3. GCP Setup
Open the Google Cloud Console and accept the terms and conditions if you are a new user. Then create a new project named RainfallCSV (you can name your own).
Creating a New Project
Creating a new project RainfallCSV
4. Set up Firestore in GCP
Once the project is created, click on the menu icon and scroll down to Storage and click FireStore.
Firestore Selection
Once the data is clicked, select the mode as Native Mode.
Selecting Native Mode
Once the Native Mode is selected, choose where to store the data (choose your nearest region) and then click Create Database.
Region Selection
Once the Database is created you will see a screen like below
Firestore Database
Now click the cloud shell icon like below
Activate Cloud Shell
After the shell is created, we will configure the project ID type the below command to check whether the project is configured or not.
gcloud config list project
Now set the project ID using the below command.
gcloud config set project PROEJCTID gcloud config set project rainfallcsv
Once the project is set, it’s time to write some logic to read CSV data and import it to Firestore.
Click the Launch Editor.
Launch Editor Click
5. Write database import code
Once the Editor is opened, create a new directory named rainfall in the terminal and then change the directory.
mkdir rainfall cd rainfall
Initialize npm using the below command in the terminal and fill the details.
npm init
Press Enter for all the options, and you should see a package.json file created with the details below.
{ "name": "rainfallcsvexport", "version": "1.0.0", "description": "Rainfall Data conversion", "main": "index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "author": "Nidhin", "license": "ISC" }
Create a new file named rainfallDataExport.js using the command in the terminal (you can name it anything).
touch rainfallDataExport.js
Copy the below code and paste in the file.
Explanation
What we have done in the above code is
Reading the contents from the CSV file and passing it to the writeToFirestore()
writeToFirestore() uses a batch process in which each of the records is processed in turn and sets a document reference based on the identifier added. At the end of the function, the batch content is written to the database.
var docRef = db.collection('rainfall').doc(record.SUBDIVISION);
6. Upload a CSV file to GCP
Now we will upload the CSV file which we are going to process. Click the link to download the Rainfall CSV File.
Upload the CSV file to the rainfall folder by right-clicking the folder name.
Uploading CSV File
Once the file is uploaded to the directory, it’s time to process the data.
7. Import the Data to Firestore
In the terminal, run the below command.
Syntax:
node javascript-filename.js csv-filename.csv
So ours will be like:
node rainfallDataExport.js rainfall.csv
Click Enter now, and you should see the data getting imported. In the console, you will get a message like below:
Wrote 4117 records
8. View the stored data in Firestore
Once you see the above message in the console, click the MenuButton -> Firestore -> Data and you should see the data imported.
Imported Data
You can view the data from the firebase console also. Go to your Firebase console and click Create a new project and then select the project name from the dropdown (the one we created in GCP console) and click Create.
Go to the Database and click Firestore you can see the data:
Data in Firebase console
Congratulations!
You have learned how to export CSV Data to Firestore using GCP.
