Which GCP Service to use - Serverless

Serverless 개념과 관련된 GCP products, USE-CASES를 정리합니다.

Which GCP Service to use - SERVERLESS GCP PRODUCTS SUMMARY

SERVERLESS = Automatically scale = High-level governance using APIs = lower Operational/Managing/Infrastructure cost

FOR APPLICATION DEVELOPMENT

Application Development - HOSTING APPLICATION USE-CASES	GCP Products
Remove the need to configure load balancers or hand manage event queues, further improving developer productivity and lowering operational overhead.	Cloud Function, App Engine
For smaller units of code that are triggered by cloud events or HTTP requests and are only billed per request, which can result in savings for applications with low or uneven traffic patterns.	Cloud Function
For larger units of code (e.g. a website) that triggered by HTTP or cloud events.	App Engine

Application Development - Stroing App data USE-CASES	GCP Products
Serverless NOSQL DB	Cloud Datastore, Firebase
Data Storage automatically scales based on traffic	Cloud Datastore, Firebase
Provides significant cost and management savings over managing your own NoSQL database such as Cassandra and MongoDB	Cloud Datastore
Additional features and functionality for mobile and web application developers : real time data synchronization across clients, integrated OAuth and Analytics	Firebase

FOR ANALYTICS

Serverless for analytics USE-CASES	GCP Products
Managed service for developing and executing a wide range of data processing patterns (ETL, batch computation & continuous computation).	Data Pipeline - Dataflow
Designed to handle unbounded, unordered, global-scale datasets (e.g. web logs, mobile usage statistics, sensor networks), which are increasingly common in day-to-day business. Server management for these dynamic resource requirements.	Data Pipeline - Dataflow
GCP’s petabyte scale, low cost data warehousing product, automatically scale out to thousands of CPUs across petabytes of data. Query data through API.	Datawarehouse -BigQuery
For organizations that want to go beyond traditional SQL-based analytics, Managed ML service based on the TensorFlow framework, providing both batch and realtime (online) prediction APIs, autoscaling depending on resource requirements, Process data on TensorFlow using GPUs and TPUs for faster training and inference.	ML service - Cloud ML Engine

FOR Microservices Architecture

Painpoints of Microservices architectures	Solution by GCP'S SERVERLESS PRODUCTS
Explosion of # of servers in Microservices architecture resulting in over provision compute resources	Reducing Infrastructure Cost : Dynamic scaling of serverless
Explosion of # of server clusters in Microservices architecture resulting in	Providing out-of-the-box self-service provisioning of servers for compute and db :Reducing Operational Cost
Common issue with microservices : Isolated architecture of services resulting in long synchronous call chains → poor performance for microservices that are customer facing.	Avoiding long synchronous call chains : (1) asynchronously “pre-computing” results for a set of microservices → (2) storing the results in a low-latency / high-throughput db. (GCP’s Cloud Pub/Sub, compute, and database services)

Serverless = No Ops

Serverless : 애플리케이션 구축에 있어 개발자가 서버 인프라에 대한 프로비져닝, 유지 및 관리를 할 필요가 없고, 사용한 만큼에 대해서만 지불하는 Cloud-native 개발 모델이다.

• No upfront provisioning, No management of servers
• pay-what-you-use economics for building applications
• Applications and databases that are not suitable to be hosted on GCP’s serverless products can be mixed and matched with services such as Compute Engine, Kubernetes Engine, or Cloud SQL.

Broad set of serverless GCP products

Client-Server 개발 초기단계부터 개발자는 Application hosting(compute) DB 와 Application DB 를 모두 관리해야했다. 효율적인 개발, 관리를 위해 구글 클라우드는 Google App Engine을 시작으로 앱 개발/분석에서 "서버리스" 프로덕트를 구축하고 있다.

• Serverless best for

  • Applications with rapid time-to-market
  • Unpredictable scale requirements (dynamically scale as needed)
  • lower infrastructure and operations costs
  • e.g, Enterprise innovation projects or startup development

Serverless GCP Products - Application "Development" and "Analytics"

1. GCP serverless Application Development

1-1. HOSTING APPLICATION

GCP SERVERLESS APPLICATION HOSTING : Cloud Functions, App Engine

Abstract the need to manage any servers & dynamically scale as needed

• App Engine : serverless platforms (PaaS)
• 2 forms of communication of Applications written with Cloud Functions & App Engine depending on the application needs
  1. Synchronous HTTP requests
  2. Asynchronous events via Pub/Sub.

Non-Serverless (lower-level services) APPLICATION HOSTING : Kubernetes Engine Compute Engine

For customers that need more control

• still some management of the underlying compute infrastructure
• requires effort on the customer's operations and development team to automate scaling.

1-2. STORING APPLICATION DATA

GCP SERVERLESS DATABASE : Cloud Datastore & Firebase

Serverless NOSQL DB & Automatically scales based on TRAFFIC

• e.g, Pokémon GO leveraged Cloud Datastore to automatically scale 50X more than the predicted traffic at launch.

Non-Serverless (lower-level services)

For customers that need more control or have special db requirements

1. install own DB on GCP VMs with persistent disk
2. GCP’s managed database products : Bigtable, Spanner, Cloud SQL.
   • offload much of the operational management overhead, but still require the customer to explicitly control scaling.

2. GCP serverless analytics : DATAFLOW, BIGQUERY, CLOUD ML ENGINE

GCP serverless analytics covers data warehousing, data pipelines (ETL), and ML.

• The backbone of these products :Cloud Pub/Sub & Google Cloud Storage allowing transfer of both streaming and batch data of any size or velocity.

GCP SERVERLESS ANALYTICS : DATAFLOW, BIGQUERY, Cloud ML Engine

• DATAFLOW : Data Pipeline
  • abstracts server management for dynamic resource requirements
  • automatically scales the underlying compute resources to meet user’s requirement on latency, correctness, and cost.
• BIGQUERY : Datawarehouse
  • low cost and high performance of Bigquery because of the ability to automatically scale out to thousands of CPUs across petabytes of data.
  • Users can interactively query data through BigQuery’s API.
• Cloud ML Engine : ML service
  • Managed ML service based on the TensorFlow framework
  • Provide both batch and online prediction API
  • automatically scales depending on resource requirements
  • Available GPUs and TPUs for training & inference

Serverless for "Microservices" Architecture

Microservices Architecture

• Require each service to have their own isolated application server cluster and database. → Require corresponding # of servers that need to run databases and application servers.
• Anti-pattern of Microservices : Sharing infrastructure

WITH GCP'S SERVERLESS PRODUCTS SOLVE PAINPOINTS OF IMPLEMENTING MICROSERVICES ARCHITECTURES.

• All of GCP’s serverless offerings provide APIs for managing serverless products, so an enterprise’s operations teams can spend their resources implementing high-level governance requirements rather than managing individual servers.
• Microservices are especially suited for serverless in terms of cost and performance.
  1. Reducing Infrastructure Cost
  2. Reducing Operational Cost
  3. Avoiding long synchronous call chains

Understanding Q

A startup plans to use a data processing platform, which supports both batch and streaming applications. They would prefer to have a hands-off/serverless data processing platform to start with. Which GCP service is suited for them?

✔ KEY : Data Processing platform - which is serverless and supports both Batch & Streaming

• ❌ Dataproc → Platform for running Apache Hadoop and Apache Spark jobs
• ❌ Dataprep → Data preparation tool based on Trifacta
• ⭕ Dataflow
• ❌ BigQuery : Data preparation tool based on Trifacta

---

Journey to Serverless on Google Cloud Platform | by Timothy | Google Cloud - Community | Medium | what-is-serverless? | Serverless Computing Solutions | Google Cloud