What You’ll Learn in This Article

・Overview of NoSQL
・Strengths, weaknesses, and use cases of various services

Introduction

In recent years, the demand for NoSQL has been increasing. This is because NoSQL is better suited for processing big data compared to relational databases. In this article, we will compare two representative NoSQL databases, **Amazon DynamoDB** and **Google Cloud Firestore**, based on the following criteria:

1. Availability

2. Functionality

3. Backup

4. Data Structure

Let’s begin with an explanation of NoSQL itself.
NoSQL is a term that refers to non-relational databases, which differ from traditional RDBMS ¹. NoSQL databases are designed to handle large-scale and complex data structures, offering excellent scalability and flexibility.
The main types of NoSQL databases include:

・Key-Value Stores

・Document Databases

・Graph Databases

・Column Databases

Key-value stores are databases that store data as key-value pairs. Both the key and value can be of any type, allowing for fast data retrieval and addition.
Document databases store data as documents. These documents are formatted as JSON, XML, or similar text formats, making data addition and modification easy.

Graph databases store data as nodes and edges. Nodes represent data elements, while edges represent relationships between nodes. These databases are suitable for data in social networks, financial transactions, and more.

Column databases store data in columns, which are contained within rows called tuples. These databases are optimized for fast data retrieval and addition.

NoSQL databases are ideal for storing and processing data that traditional RDBMS cannot handle. They are commonly considered for applications dealing with large-scale or complex data structures.

What is Amazon DynamoDB?

Amazon DynamoDB is a fully managed NoSQL database service provided by AWS. Unlike traditional RDBMS, DynamoDB pricing is based on throughput rather than data size.
DynamoDB excels in scalability, durability, and availability. It automatically scales, prevents data loss, and ensures high availability.
DynamoDB is used in various applications, including web applications, mobile apps, and IoT applications, making it suitable for handling large-scale data and complex data structures.

What is Google Cloud Firestore?

Google Cloud Firestore is a fully managed NoSQL database service offered by Google Cloud Platform. Firestore is a document-based database that stores JSON documents. It is scalable, durable, and user-friendly.
Like DynamoDB, Firestore is used in a wide range of applications, including web, mobile, and IoT. Furthermore, Firestore integrates seamlessly with other Google Cloud services, such as Cloud Functions, Cloud Storage, and Cloud Pub/Sub.

Comparison

Availability

• DynamoDB automatically replicates data across multiple AZs (Availability Zones) within the same region ².
• Firestore also uses region replication to replicate data across multiple AZs in the same region. For multi-region configurations, data is replicated to at least two different regions. However, multi-region support is limited to regions in Europe and the United States, making it less practical for use in Japan.

While both offer excellent availability, DynamoDB’s automatic replication is more user-friendly.

Functionality

• DynamoDB responds to requests in milliseconds, regardless of dataset size.
• Firestore prioritizes high availability and durability, so it does not provide response times as fast as DynamoDB.

In terms of functionality, DynamoDB is the recommended choice.

Backup

• DynamoDB offers two backup methods. The first uses AWS Backup to back up all tables. The second employs Point-in-Time Recovery, allowing users to restore tables to any state within the past 30 days.
• Firestore requires users to create an import/export workflow using Google DataStore. This workflow is complex and challenging to set up.

For backup options, DynamoDB is more favorable.

Data Structure

• DynamoDB uses a key-value data model, storing data as key-value pairs.
• Firestore is a document database that stores data as JSON documents.

The key-value model of DynamoDB is more basic, making it easier to learn. However, document databases like Firestore offer flexibility in storing complex data structures, making them suitable for advanced designs.

Use Cases

DynamoDB

• Game state and player session data storage

Firestore

• Offline-first applications and real-time applications

Conclusion

This article compared Amazon DynamoDB and Google Cloud Firestore from four perspectives.
In terms of availability, functionality, and backup options, DynamoDB is the preferred choice. However, if you prioritize complex data design, Firestore’s document-based model is advantageous.
Consider starting with DynamoDB to familiarize yourself with NoSQL. Once comfortable, try Firestore for projects requiring complex data designs.

Glossary

⁴ indicates independent content and should remain unchanged.[/efn_note]

What You Can Learn from This Article

– Overview of Lambda, ECS, and Batch – Serverless concepts – Container concepts – Pros and cons of each service and their use cases

Introduction

AWS offers a wide variety of computing services, which can be great for application development but may be confusing for beginners. This article will explain three AWS services: AWS Lambda, AWS ECS, and AWS Batch, from the perspectives of:

1. Cost performance

2. Flexibility

3. Functionality

We will dive into the details of these services. To fully understand the content of this article, some knowledge of serverless and container services is required. So, let’s first explain what serverless and container services are.

What is a Serverless Service?

A serverless service is a cloud computing service where the service provider manages the server hardware and infrastructure. Users only use the resources such as server instances and storage capacity as needed, and are charged based on usage. Serverless services have several advantages:

・No need for infrastructure management, which helps reduce costs.

・Development can focus on core tasks, increasing productivity.

・Excellent scalability, capable of handling sudden increases in traffic.

Serverless services are used in various fields such as web application development, mobile app development, data analysis, and IoT. Some representative serverless services include:

・AWS Lambda (introduced in this article)

・Microsoft Azure Functions

・Google Cloud Functions

・IBM Cloud Functions

Serverless services are a growing trend in cloud computing and are expected to become even more widespread.

(For more about serverless services, check here↓)

Which serverless service should you use? AWS Lambda vs Azure Functions vs Cloud Functions

Which serverless service should you use on AWS? AWS Lambda vs AWS Fargate

What is Container Service?

A container service refers to a cloud service that utilizes container technology to deploy, run, and manage applications. Containers are lightweight virtual units that package applications along with their dependencies and runtime environments. By using container services, applications can be deployed more efficiently, scalability is improved, and management becomes easier. Specific benefits include:

・Containers help reduce issues caused by environment discrepancies since the application’s environment is encapsulated within the container.

・Containers are lightweight, start quickly, and allow for efficient resource utilization.

・Scalability is enhanced, enabling flexible response to load fluctuations.

・Containers are ideal for implementing microservices architecture and building loosely coupled applications.

However, when adopting container services, attention should be paid to security, network settings, monitoring, and management. Proper design and best practices are crucial.
Some representative container services include Docker and Kubernetes.

Docker

Docker is a pioneer of container technology, allowing applications and their dependencies to be packaged into containers, fully isolating the runtime environment. Many cloud providers (AWS, Azure, Google Cloud, etc.) offer managed services for hosting Docker containers, which makes it easy to deploy, scale, and monitor containers.

Kubernetes

Kubernetes is a container orchestration system that allows for the deployment, scaling, automatic recovery, and version management of multiple containers. Kubernetes is an open-source project widely used by various cloud providers and in on-premises environments. It is highly flexible and scalable, making it suitable for deploying various types of applications.

What is AWS Lambda?

AWS Lambda is a serverless computing service provided by AWS. It allows you to execute code without worrying about servers or clusters. AWS Lambda provides an environment to run code when events occur. Events can include HTTP requests, message queues, database changes, and more. AWS Lambda can be used for web applications, backend services, data analytics, and other purposes.
Benefits of AWS Lambda include:

・No need to manage servers or clusters

・Scalable and automatically runs code each time an event occurs

With AWS Lambda, there’s no need to manage servers or clusters, making it easier to handle server load.

What is Amazon ECS?

Amazon ECS is a fully managed service for running and managing Docker containers, often used in conjunction with Amazon EC2 ¹. ECS allows you to create, deploy, scale, and manage containers. It can be used for various purposes such as web applications, backend services, and data analytics.
Benefits of ECS include:

・Easy container creation, deployment, scaling, and management

・Scalable, allowing automatic adjustment of container numbers based on load

With ECS, container creation, deployment, scaling, and management are simplified, allowing for efficient container operation.

What is AWS Batch?

AWS Batch is a service for running scalable and simple batch processing jobs. AWS Batch allows you to create, schedule, and execute batch processing jobs. It supports automatic scaling of compute instances, adjusting the number of instances according to the batch processing job’s load.
Benefits of AWS Batch include:

・Scalable and simple execution of batch processing jobs

・Supports automatic scaling of compute instances, adjusting based on job load

With AWS Batch, you can run scalable and simple batch processing jobs, efficiently managing your tasks.

Comparison

Availability

• With Lambda, if memory runs out, you can increase the memory amount from the console, and it can handle sudden spikes in requests without issues.
• With ECS, if memory runs out, you cannot increase memory from the console. You need to launch a new instance and set up an environment to run your program again.
• Batch is the same as ECS in this regard.

In terms of availability, Lambda is better.

Flexibility

• Lambda has a limit of 15 minutes for execution time, and the supported development languages are limited to nodeJS, Python, Go, Java, Ruby, .Net, and C#. However, most necessary languages are available for development.
• ECS and Batch allow you to freely select server specifications based on your needs.

In terms of flexibility, ECS and Batch are superior.

Functionality

• Lambda forces a stop after 15 minutes of processing.
• ECS allows processing for more than 15 minutes.
• Batch allows processing for more than 15 minutes and is suitable for larger-scale processing than Amazon ECS.

In terms of functionality, ECS and Batch are recommended.

Cost

• Lambda charges based on usage time and has a free tier of 1 million requests per month.
• ECS charges based on the time EC2 instances are running.
• Batch charges for the compute resources used.

These services have different pricing structures, so it’s difficult to compare them directly. Here’s an example of costs with specific conditions:

・Process running with 128MB memory

・Each API call takes 3 seconds

・1 request per second

Lambda Cost Calculation

・Requests 86,400 (requests/day) * 30 (days) = 2,592,000 requests 2,592,000 – 1,000,000 (free tier) = 1,592,000 requests 1,592,000 / 1,000,000 * 0.20 USD = 0.3184 USD ・Execution time 0.128 (GB/request) * 3 (seconds) = 0.384 (GB-seconds/request) 0.384 * 86,400 (requests/day) * 30 (days) = 995,328 GB-seconds 995,328 – 400,000 (free tier) = 595,328 GB-seconds 595,328 * 0.0000166667 USD = 9.9222 USD
Total: 0.3184 + 9.9222 = 10.2406 USD

EC2 Cost Calculation

For a t3.nano instance (2 vCPUs, 0.5GiB memory): 0.0068 USD/hour * 24 (hours) * 30 (days) = 4.896 USD

AWS Batch

For a t3.nano instance with 2 vCPUs and 0.5 GiB of memory:
Calculation:
0.0068 USD/hour * 24 hours * 30 days = 4.896 USD

Use Cases

AWS Lambda is typically used for small-scale processing, often combined with other services. Specifically, it is frequently used with:

• API Gateway to create API systems that simplify the management and execution of APIs.
• Amazon EventBridge for cron systems, enabling automated tasks like nightly database maintenance or hourly sales data aggregation.
• SQS for asynchronous processing, such as credit card payment processing in e-commerce sites.

Amazon ECS is used for large-scale processing that takes a long time. It is particularly suited for Microservices²,Batch jobs, where tasks are pre-defined and executed in bulk.

Batch is used for batch processing that requires a large number of parallel executions beyond the service quotas of ECS. ³It is typically used in fields such as life sciences, computational chemistry, clinical modeling, and financial services for processing transaction data or analyzing sensitive information. AWS Batch is considered over-spec for general corporate use and is mainly employed by large enterprises and research institutions.

Conclusion

This article explained the AWS computing services: AWS Lambda, Amazon ECS, and AWS Batch. For beginners, Lambda is easier to use, and AWS Batch may be overkill for typical development tasks. If you’re new to AWS development, start with Lambda and once you’re comfortable, try using ECS as well.

Explanation of Terms