Overview

In this post, you will learn how to import CSV data to a Firestore database using GCP.

Objectives

• What is GCP
• What is Firestore
• GCP Setup
• Set up Firestore in GCP
• Write database import code
• Upload a CSV file to GCP
• Import the data to Firestore
• View the stored data in Firestore

1. What is GCP?

Google Cloud Platform (GCP) is a suite of public cloud computing services provided by Google. The platform includes a range of hosted services for compute, storage, and applications that run on Google Hardware.

Google Cloud Platform services can be accessed by software developers, cloud administrators, and other enterprise IT professionals over the public internet or through a dedicated network connection.

2. What is Firestore?

Cloud Firestore is a fast, fully managed, serverless, cloud-native NoSQL document database that simplifies storing, syncing, and querying data for your mobile, web, and IoT apps on a global scale.

Its client libraries provide live synchronization and offline support, while its security features and integrations with Firebase and Google Cloud Platform (GCP) accelerate building truly serverless apps.

3. GCP Setup

Open the Google Cloud Console and accept the terms and conditions if you are a new user. Then create a new project named RainfallCSV (you can name your own).

Creating a New Project

Creating a new project RainfallCSV

4. Set up Firestore in GCP

Once the project is created, click on the menu icon and scroll down to Storage and click FireStore.

Firestore Selection

Once the data is clicked, select the mode as Native Mode.

Selecting Native Mode

Once the Native Mode is selected, choose where to store the data (choose your nearest region) and then click Create Database.

Region Selection

Once the Database is created you will see a screen like below

Firestore Database

Now click the cloud shell icon like below

Activate Cloud Shell

After the shell is created, we will configure the project ID type the below command to check whether the project is configured or not.

gcloud config list project

Now set the project ID using the below command.

gcloud config set project PROEJCTID
gcloud config set project rainfallcsv

Once the project is set, it’s time to write some logic to read CSV data and import it to Firestore.

Click the Launch Editor.

Launch Editor Click

5. Write database import code

Once the Editor is opened, create a new directory named rainfall in the terminal and then change the directory.

mkdir rainfall
cd rainfall

Initialize npm using the below command in the terminal and fill the details.

npm init

Press Enter for all the options, and you should see a package.json file created with the details below.

{
"name": "rainfallcsvexport",
"version": "1.0.0",
"description": "Rainfall Data conversion",
"main": "index.js",
"scripts": { "test": "echo \"Error: no test specified\" && exit 1" },
"author": "Nidhin",
"license": "ISC"
}

Install Dependencies

Install the following dependencies.

npm install @google-cloud/firestore
npm install csv-parse

Create a new file named rainfallDataExport.js using the command in the terminal (you can name it anything).

touch rainfallDataExport.js

Copy the below code and paste in the file.

Explanation

What we have done in the above code is

• Reading the contents from the CSV file and passing it to the writeToFirestore()
• writeToFirestore() uses a batch process in which each of the records is processed in turn and sets a document reference based on the identifier added. At the end of the function, the batch content is written to the database.

var docRef = db.collection('rainfall').doc(record.SUBDIVISION);

6. Upload a CSV file to GCP

Now we will upload the CSV file which we are going to process. Click the link to download the Rainfall CSV File.

Upload the CSV file to the rainfall folder by right-clicking the folder name.

Uploading CSV File

Once the file is uploaded to the directory, it’s time to process the data.

7. Import the Data to Firestore

In the terminal, run the below command.

Syntax:

node javascript-filename.js csv-filename.csv

So ours will be like:

node rainfallDataExport.js rainfall.csv

Click Enter now, and you should see the data getting imported. In the console, you will get a message like below:

Wrote 4117 records

8. View the stored data in Firestore

Once you see the above message in the console, click the MenuButton -> Firestore -> Data and you should see the data imported.

Imported Data

You can view the data from the firebase console also. Go to your Firebase console and click Create a new project and then select the project name from the dropdown (the one we created in GCP console) and click Create.

Go to the Database and click Firestore you can see the data:

Data in Firebase console

Congratulations!

You have learned how to export CSV Data to Firestore using GCP.

Happy Learning :)