BigQuery community UDFs go global to simplify data transformations for everyone

1 month ago 17
News Banner

Looking for an Interim or Fractional CTO to support your business?

Read more

BigQuery users, rejoice! We're excited to announce that community-contributed BigQuery User Defined Functions (UDFs) from the popular bigquery-utils repo are now available in all BigQuery regions. This expansion means that no matter where you're running your data warehouse workloads, you can leverage these powerful tools to streamline your data transformations.

What are UDFs and why should you care?

UDFs are custom functions you can create in BigQuery to handle specific tasks. They work like built-in SQL functions but are tailored to your unique needs. Need to parse complex strings? Want to perform calculations not readily available in standard SQL? UDFs are your answer.

Previously, community-contributed UDFs were publicly available but limited to the US multi-region in the bqutil project. This meant users outside of the US had to manually deploy UDFs to their own regional dataset within their own project, adding extra steps to their workflows. Today, we're eliminating this limitation by opening up community-contributed BigQuery UDFs to the world.

Why is this expansion important for customers

  • Global accessibility: No matter where your BigQuery data resides, you can now tap into the vast collection of community-developed UDFs. This truly democratizes access to advanced data transformation techniques.

  • Community collaboration: This expansion is a testament to the power of open-source collaboration. It demonstrates how Google Cloud and the wider community are working together to make BigQuery the most accessible and versatile data warehouse solution.

How to get started

Using UDFs in your region is simple. Just reference the UDFs with a regional suffix:

code_block <ListValue: [StructValue([('code', 'bqutil.<dataset>_<region>.<function>()'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3dfcd3f9cd60>)])]>

For example, here's how to use the typeof() function in different regions:

code_block <ListValue: [StructValue([('code', 'bqutil.fn_eu.typeof() -- eu multi-region\r\nbqutil.fn_europe_west1.typeof() -- europe-west1 region\r\nbqutil.fn_asia_south1.typeof() -- asia-south1 region'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3dfcd3f9cdc0>)])]>

Learn more and contribute

To explore the full range of available UDFs and learn more about how to use them, check out the bigquery-utils GitHub repository.

Are you skilled at creating user-defined functions? We welcome you to share your own functions with the community. Please see our contributing documentation to get started. Let us know how you're putting these global UDFs to work! Happy querying!

We'd like to express our sincere gratitude to the maintainers and contributors of the bigquery-utils project - including everyone who has dedicated their time and expertise.

Read Entire Article