BigQuery on Lin's Blog

BigQuery SQL String Split

Sun, 17 Sep 2017 13:39:48 +0000

Most programmers are familiar with split. It’s one of the ten most commonly used functions in programming. BigQuery has a split function too, but since it’s SQL, the usage is quite different from general-purpose programming languages. On top of that, BigQuery comes in two versions, Standard SQL and Legacy SQL. Both have split, but they are used in completely different ways.

BigQuery Federated Google Sheets Table Access Denied Error

Thu, 14 Sep 2017 21:45:47 +0000

BigQuery lets you link a table to a Google Sheet stored in Google Drive, so the table updates right away whenever the Google Sheet changes. When you use this kind of federated table, some people run into an Access Denied error. Queries in the WebUI work fine, but when you query the table with the bq command line tool, you get the following error message:

Access Denied: BigQuery BigQuery: No OAuth token with Google Drive scope was found

BigQuery SQL Get First or Last Day of Month From a Specified Date

Sat, 29 Jul 2017 18:38:56 +0000

In BigQuery, if you want to get the first day from a date (or a timestamp or datetime value), for example turning “2017-05-24” into “2017-05-01”, you can call the UTC_USEC_TO_MONTH function in Legacy SQL, or DATE_TRUNC() in Standard SQL. In fact, BigQuery has a whole series of functions that do these things, getting the first day, hour, week, and so on from a date.

BigQuery SQL Get Random N Row Sampling

Fri, 28 Jul 2017 23:57:37 +0000

When doing data analysis, randomly picking N rows from a dataset (sampling) is a very common need. You might sample to check whether the data is correct, or sample so a model can be built when there is too much data to process comfortably. Sampling with SQL inside the database is actually very simple. Here I’ll demonstrate with BigQuery Standard SQL, but Legacy SQL works the same way.

Dragon Boat Festival Holiday: Saying Goodbye to Traffic Jams - Yilan Edition

Fri, 10 Jun 2016 14:29:03 +0000

It’s been nearly a year since my last article, and I’m finally publishing a new one! During the 2016 Dragon Boat Festival holiday, I’m sure many friends from Taipei will choose Yilan as their first destination. Whether you drive or take a bus, if you go through the Xueshan Tunnel on National Freeway No. 5, it takes only about 40 minutes to reach Yilan if all goes well. Of course, everyone knows that during a holiday like this, things are rarely that smooth… Today I’m going to analyze the traffic patterns of the Xueshan Tunnel so you can avoid the crowds and get to Yilan smoothly! Below there will be a short technical explanation and the analysis process; if you’re not technically inclined, you can jump straight to the conclusion! By the way, I’m in Yilan today, and this article was actually written at the Brown Cafe Castle.