Private DMPs are popular these days, but most of them are treasure data (YBI). Treasure data is very useful (especially td-js-sdk), but it's a bit expensive.
On the other hand, BigQuery is attractive for its low price and query execution speed, but it is very inconvenient to input and output data compared to treasure data.
So, I made an application with GAE (Python) that realizes ease of use like treasure data on BigQuery. If you use your own source code, you can build it with zero development.
Source code: https://github.com/mats116/ElasticBigQuery
--It works just like treasure data just by changing the endpoint as shown below.
--writeKey is static at the moment, but we plan to make it possible to issue and manage permissions from the screen.
--Only JSONP, which has become the default since v1.4.0, is supported.
--BigQuery is not schemaless, but it automatically identifies it in the API and automatically generates datasets and tables.
--The generated table name has a date (UTC) at the end, such as table_id + YYYYMMDD
, making it a daily table.
--As a unique function, the following parameters are acquired in the URL-decoded state.
- td_path
- td_referrer
- td_url
pageviews.js
<script type="text/javascript">
var td = new Treasure({
host: 'elasticbigquery.appspot.com',
pathname: '/dmp/v1/event/',
writeKey: 'thie_is_static_setting_yet',
database: '<dataset_id>'
});
td.trackPageview('<table_id>');
</script>
--Returns a transparent GIF with the endpoint below.
- //elasticbigquery.appspot.com/dmp/v1/beacon/<dataset_id>/measurement
--You can get the GET parameter.
--There are some parameters such as referrer that are acquired by default.
--Issue bqid
under the GAE domain xxx-xxx.appspot.com
.
--Please change the domain and id name according to the purpose.
--You can check it at the following endpoint.
- http://elasticbigquery.appspot.com/dmp/v1/bqid/get
--If you add callback = hoge
to the GET parameter, you can use it like JSONP.
--I want to be able to grant permissions for BigQuery datasets to the account signed up with Oauth (Google) from the UI.
--I want to be able to issue a writeKey from the UI. (Currently static) ――It's hard to refer to Cloud Datastore every time, so maybe memocache?
--I want to be able to export query results to S3 and Google Cloud Storage. ――I wonder if it can be integrated with Apps Script in a nice way
--Completely freeride to td-js-sdk. .. ――When you fail to put it in BigQuery, it is loaded in the Task queue, so it is made quite firmly. ――Because it is a personal project, we are looking for a company that can do GAE load verification. ――I'm not good at making UI, so please help me.
For your reference. If you are new to GAE, do your best.
--Google account (@gmail address is also OK) -PC with Google App Engine SDK
-Create a new project in the Google Developers Console. --App Engine location is ** us-central ** is the closest to Japan --By default ** BigQuery API ** should be enabled, but just in case
--Since it is published on GitHub, please clone it appropriately. - https://github.com/mats116/ElasticBigQuery
--Open ʻapp.yaml` and rename the project --In terms of source code, here
--Deploy from Google AppEngine Launcher.
Recommended Posts