Do you want to deploy Trino on Dataproc easily? Are you searching for Trino initialization script? Here it is; Download “trino.sh” from github and upload it to your GCS bucket. Here is my github link https://github.com/sametkaradag/initialization-actions/blob/master/trino/trino.sh which is fork of https://github.com/GoogleCloudDataproc/initialization-actions (waiting for the pull as of writing this post)

Do you need to fill a BigQuery table with random set of numbers, dates…? cool go on.. BigQuery has a cool Generate_Array function which gets the range bounds as inputs and generates an array (nested field). Generate_date_array is the equivalent for generating date arrays. Unnest function can be used to…

Prerequisites: You need terraform Refer to https://learn.hashicorp.com/tutorials/terraform/install-cli for installing terraform. 1- Create tfvar file: #cat terraform.tfvars project = "project-id" region = "europe-west4" 2- Create tf file and put mysql database related information in it, such as instance, database name and root password: #cat main.tf variable "project" {} variable "region" {} …

Duplicate data sometimes can cause wrong aggregates or results. You probably need to remove those duplicate rows before doing any aggregation, join or calculation. …

Want to migrate Oracle Database to Big Query? Don’t know how to move data from Oracle to Big Query? Want to do this with GCP native tools? Keep reading, you are on the right post! Here are some advantages using Dataproc and Sqoop for your Oracle migrations; Dataproc will be…

Assumptions The backup strategies are based on the following assumptions: Data size in BigQuery: 50TB Data Ingestion Method: Append only (Load data into new daily partitions, preserving all the historical data in older partitions) Load frequency: Daily Current Data Retention in BQ: 2 years of data stored without any deletes or…

TL:DR; gsutil -m -o GSUtil:parallel_composite_upload_threshold=150M cp -r local_dir gs://your-bucket/data/ You need to create a bucket in your GCP project: There are 3 important things here; Location type: region, multi-region, dual-region (affects pricing) Location of the bucket: eu for multi-region or one of the options below for regional locations within Europe: …