Would you like to influence Google Cloud’s BI roadmap more? I’d like to invite you to join the Google Cloud BI...
from last 7 days Dataproc serverless batch job failing with below errorConfiguring index-url as 'https://us-py...
There are 3 Dataproc optionsDataproc on GCEDataproc with GKEDataproc ServerlessWhat are the pros/cons of each,...
I am trying to calculate the Cost of Dataproc and then Display on Looker Report and also trying to automate th...
Hi,I'm using Dataproc on VMs. Currently we are running in production two clusters: master VM typeworker VM typ...
Hi all, I've been trying to connect to an Iceberg table that I created through a bigquery stored procedure. My...
In Log Explorer, run the following query for the last 30 days got 51 entries.resource.type="cloud_dataproc_clu...
I created an autoscaling policy and attached to a cluster with 2 primary worker nodes (2 core with 8gb). I ran...
@jangemmarPer doc: "https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs#State"0: ST...
I was following the example https://cloud.google.com/pubsub/lite/docs/samples/pubsublite-spark-streaming-from-...
Hi, I was added to a group to access the google cloud skill boost credits owned by a person. I received mail r...
Hi,I am currently converting some Alteryx workflows into pyspark and wanted to test the codes for debugging an...
Hi all,I've been struggling with it for the past 2 weeks, going through GCP documentation and looking of the s...
Hello,I am currently tasked with analyzing the historical resource consumption of our Dataproc ephemeral clust...
Hello all,I've successfully set up an Iceberg table with a BigLake Metastore as per this documentation:https:/...
I need help to read data from hudi using trino in dataproc cluster.
is there a way to speed this up? the script itself is very fast, but the action of actually calling the stored...
Hi,I want to run a PySpark stored procedure using a custom container image, but when I do, I receive the follo...
my quota is 4096 GB and its reached after I triggered the serverless spark job ( which failed) but its still s...
Hello, I am unclear about the Google-Managed Encryption Process.- How often does google rotate encryption keys...
I have a Flink job running in a dataproc cluster. It's exporting logs to Log Explorer but I don't think Log Ex...
custom/cpu/utilizationcustom/disk/percent_usedhow are the above cloud dataproc batch metric useful when settin...
I need help regarding creating Dataproc alerts.What all type of alert we can create for Dataproc.
Hi,I have a Dataproc cluster and using the web interface (Jupyterlab) inside the cluster, I am trying to read ...
I noticed this alert on the serverless batches page:"Starting March 31, 2024, special requirements apply to us...
Hi Folks,I'm trying to submit to submit a Dataproc serverless job using service account (have necessary permis...
Hi, I have a pipeline set up, it creates a cluster perform an ingestion and delete a cluster. In success logs ...
Hi,we have created a metastore and it has a database and table, post editing the metastore and enabling the Da...
When trying to use Dataproc image version 2.2. the gcloud CLI raises a validation error.Trying the same comman...
Hi,we're using hive tables with dataproc metastore synced to data Catalog, we could see that data catalog was ...
I have a dataproc cluster with livy installed and it is working.what is the command to run a 'hello world' scr...