Gemini platform

Gemini platform is GCP's unified ML platform. It brings together training, prediction, Feature Store, pipelines, model monitoring, and experiment tracking under one service.

Training Options (Decision Tree)

• BigQuery ML: Train models using SQL directly in BigQuery. Best for analysts who know SQL and need quick models (linear regression, logistic regression, k-means, ARIMA, boosted trees, DNN). No data movement needed.
• AutoML: No-code training for tabular, image, text, and video data. Best when you want good performance with minimal ML expertise. Handles feature engineering, architecture search, and hyperparameter tuning automatically.
• Custom Training: Full control using TensorFlow, PyTorch, scikit-learn, or XGBoost in custom containers. Best when you need specific architectures, custom loss functions, or specialized preprocessing.
• Pre-trained APIs: Vision AI, Natural Language AI, Speech-to-Text, etc. Best when the task matches a pre-trained capability and you don't need custom models.

Same need for a trained model, four different paths — click (or tab to) any node to see why.

SQL-familiar analysts, quick model:

1. Data BigQuery table

   Data already lives in BigQuery — no need to move it anywhere to train on it.

2. Train BigQuery ML

   Chosen because: the team knows SQL, not Python or ML frameworks, and needs a model fast — BigQuery ML trains in place with CREATE MODEL, no data movement.

Minimal ML expertise, good performance:

1. Data Tabular / image / text / video

   Any of these data types works — AutoML doesn't require the data to already live in BigQuery.

2. Train AutoML

   Chosen because: it handles feature engineering, architecture search, and hyperparameter tuning automatically — the right call when the team wants good results without ML expertise.

Custom architecture or loss function:

1. Data Any format

   Custom Training imposes no data-format requirement — that's the point of full control.

2. Train Custom Training

   Chosen because: TensorFlow, PyTorch, scikit-learn, or XGBoost in a custom container give you the specific architecture, loss function, or preprocessing that BigQuery ML and AutoML can't.

   Instead of AutoML: which only tunes within its own supported model types — not a fit when the architecture itself is the requirement.

Task matches an existing capability:

1. Data Image / text / speech

   No training data needed at all — that's what distinguishes this path from the other three.

2. Infer Pre-trained APIs

   Chosen because: Vision AI, Natural Language AI, and Speech-to-Text are already trained — skip the other three paths entirely when the task matches one of these out of the box.

Gemini platform Feature Store

A centralized repository for organizing, storing, and serving ML features. Solves the critical problem of training-serving skew and feature reuse across teams.

Key Capabilities

• Feature sharing: Multiple teams can register and discover features, avoiding duplicate computation.
• Online & offline serving: Online serving for low-latency predictions; offline serving (BigQuery) for batch training and batch predictions.
• Point-in-time lookups: For training data, Feature Store can return feature values as they were at a specific timestamp — preventing data leakage.
• Feature monitoring: Tracks feature distributions over time and alerts on drift, staleness, or anomalies.
• Freshness monitoring: Detect when feature pipelines are delayed and features become stale.

Model Deployment & Serving

Gemini platform Endpoints

• Online prediction: Low-latency, real-time predictions via REST or gRPC. Supports autoscaling, traffic splitting (for A/B testing), and canary deployments.
• Batch prediction: Process large datasets asynchronously. Input from GCS or BigQuery, output to GCS or BigQuery. Cost-effective for bulk scoring.

Model Monitoring

• Feature skew detection: Compares training data distributions with incoming prediction request distributions.
• Prediction drift: Detects when prediction output distributions shift over time, suggesting model decay.
• Alerting: Configure thresholds and notification channels for automated alerts.

MLOps with Gemini platform Pipelines

Gemini platform Pipelines orchestrate ML workflows using Kubeflow Pipelines or TFX. They define reproducible, end-to-end ML workflows: data prep → training → evaluation → deployment.

• Pipelines are defined as DAGs of components
• Each component runs in a container with tracked inputs/outputs
• Metadata (parameters, metrics, artifacts) is automatically logged to Vertex ML Metadata
• Supports conditional execution (deploy only if accuracy exceeds threshold)

BigQuery ML

Train and serve models using SQL, directly in BigQuery. No data movement, no separate infrastructure.

Supported Model Types

CREATE OR REPLACE MODEL `project.dataset.fraud_model`
OPTIONS(
  model_type='BOOSTED_TREE_CLASSIFIER',
  input_label_cols=['is_fraud'],
  auto_class_weights=TRUE,
  data_split_method='AUTO_SPLIT'
) AS
SELECT * FROM `project.dataset.training_features`
WHERE transaction_date BETWEEN '2024-01-01' AND '2024-12-31';