Skip to content

Event storage structure

Overview

You can check the structure in which sent event logs are stored in BigQuery, the detailed structure of the attributes JSON field, and how logs that violate the transmission rules are isolated.

Table structure

Sent event logs are stored in the t_raw_event_log table of each company's BigQuery dataset.

Table column structure

Storage location

Item Format
Table t_raw_event_log
Partitioning Daily partitions based on dateTime

Column definitions

Column name Data type Required Description Example
userId STRING Y Account-based user identifier 100001
identifierProvider STRING Y System that issues the user identifier. Always hive hive
deviceId STRING Y Device-based user identifier 100000
appId STRING Y App ID registered in Hive Console > Project Settings > App ID "com.com2us.game.ios"
appIdGroup STRING Y Project ID issued in Hive Console > Project Settings > Project. Value converted based on appId com.com2us.game
dateTime TIMESTAMP Y Time when the event occurred. Value of eventTime converted to UTC 2026-07-20T05:01:01Z
eventName STRING Y Event name that classifies the event asset_drop
checksum STRING N Hash value generated only from the original data that the customer sent, excluding all fields injected by the pipeline or generated by the system (such as hiveAttributes) HW2FTEB56TEEWER
bigqueryRegistTimestamp TIMESTAMP N UTC time when the event was stored in BigQuery 2026-07-20T05:02:01Z
attributes JSON N JSON field that stores custom attributes and automatically collected attributes See below

Table structure notes & tips

  • The same event can be stored more than once because of resending or other causes, so we recommend that you also use a deduplication query based on checksum when you query the data.

    QUALIFY ROW_NUMBER() OVER (PARTITION BY checksum ORDER BY bigqueryRegistTimestamp DESC) = 1

  • Event logs that violate the transmission rules are stored in the rescue table, not in this table. For details, see Data isolation.

attributes details

Note

The attributes column is of JSON type and is divided into two areas: hiveAttributes and eventAttributes.

Example of attributes

{
  "hiveAttributes": {
    "_clientIp": "220.70.82.0",
    "_dataSource": "custom_server",
    "_fluentd_appid": "com.com2us.game.ios",
    "_fluentd_sender_ip": "43.101.101.33",
    "_fluentd_hostaname": "9104813d31ab",
    "_fluentd_tag": "ha2union.game.stage",
    "_geoIpCountry": "KR",
    "_market": "google",
    "_os": "android"
  },
  "eventAttributes": {
    "eventTime": "2026-07-20T14:01:01+09:00",
    "level": 10,
    "character_name": "AA",
    "stage_id": "stage_101"
  }
}

hiveAttributes

This is the area that the pipeline processes and stores automatically.

Attribute name Data type Description Example
_clientIp STRING If client_ip or clientIp is sent with the log, the sent IP with its last octet changed to 0. Not created if client_ip or clientIp is not sent 220.70.82.0
_geoIpCountry STRING Country code that the pipeline determined based on client_ip or clientIp (ISO 3166 alpha-2). Not created if client_ip or clientIp is not sent KR
_dataSource STRING Path through which the event log actually came in, as determined by the pipeline. See the _dataSource details below custom_server
_fluentd_appid STRING appId of the log collector (Fluentd) as determined by the pipeline com.com2us.game.ios
_fluentd_sender_ip STRING IP of the log collector (Fluentd) as determined by the pipeline 43.101.101.33
_fluentd_hostaname STRING Hostname of the log collector (Fluentd) as determined by the pipeline 9104813d31ab
_fluentd_tag STRING Name of the tag sent to the log collector (Fluentd) as determined by the pipeline ha2union.game.stage
_market STRING Market information that Analytics determined from the appId google
_os STRING OS information that Analytics determined from the appId android

_dataSource details

_dataSource indicates the path through which the event log actually came in, and one of the following values is set automatically.

Value Description
airbridge Events collected through Airbridge
appsflyer Events collected through Appsflyer
adjust Events collected through Adjust
singular Events collected through Singular
axyl_server Events collected automatically through Hive Axyl authentication, billing, and so on
custom_server Custom events that the customer sends from a server
axyl_custom_sdk Custom events that the customer sends through the event log sending feature of the Hive Axyl SDK
custom_except Events sent with eventAttribute and hiveAttribute generated as an exceptional case during event conversion

eventAttributes

This is the area that stores the attributes you sent and the attributes the SDK collected automatically.

Attribute name Data type Description
eventTime STRING Original eventTime value sent as a required attribute
(custom attributes) Number/text Custom attributes that you sent directly
(automatically collected attributes) Varies by attribute Client information that the SDK collected automatically (when sent from the client)

attributes notes & tips

  • hiveAttributes is not an area where the developer sets values directly; the pipeline fills it automatically. Values other than _clientIp, _dataSource, _fluentd_appid, _fluentd_sender_ip, _fluentd_hostaname, _fluentd_tag, _geoIpCountry, _market, and _os are not stored.
  • Custom attributes and attributes that the SDK collects automatically are both stored together in eventAttributes. When you query them, you must distinguish them by attribute name.
  • You can use the _dataSource value to query event logs separately by transmission path (such as SDK or server).

Data isolation

Warning

Event logs that do not follow the transmission rules are not stored in the normal table (t_raw_event_log). Instead, they are stored separately in the isolation table (t_raw_event_log_rescue). Because the isolation table also keeps the original JSON, you can identify the cause of the isolation, fix it, and resend the event log.

Isolation reasons

Error reason Description How to handle
Missing required attribute Any of the seven required attributes (userId, identifierProvider, deviceId, appId, appIdGroup, eventTime, eventName) is missing or empty Resend with the missing required attributes included
Reserved word used attributes, eventAttributes, or hiveAttributes is used as an attribute name (case-insensitive) Change the attribute name to a different name
Disallowed attribute value The attribute value is a JSON object ({}) or a JSON array ([]) Change the attribute value to a string or a number
eventTime format error eventTime is not in RFC 3339 format or cannot be parsed Change it to RFC 3339 format (including the time zone)
eventTime out of range eventTime is outside the range of 10 years in the past to 5 years in the future relative to the current time Check the eventTime value and change it to fall within the valid range
JSON parsing failure The sent message is not in valid JSON format Validate the JSON format and resend

Isolation table structure

Isolated event logs are stored in the t_raw_event_log_rescue table with the following structure. Each column is filled only when the original JSON can be parsed; otherwise, it is stored as an empty value. You can check the original in rawData.

Column name Data type Description
userId STRING userId as sent
identifierProvider STRING identifierProvider as sent
deviceId STRING deviceId as sent
appIdGroup STRING appIdGroup as sent
dateTime STRING Original eventTime value as sent (stored as a string as is, not TIMESTAMP)
eventName STRING eventName as sent
bigqueryRegistTimestamp TIMESTAMP Time when the log was stored in BigQuery
checksum STRING Hash value generated only from the original data that the customer sent, excluding all fields injected by the pipeline or generated by the system (such as hiveAttributes)
rawData STRING Entire original JSON as sent
errorReason STRING Isolation reason code. See Isolation reason codes
errorStackTrace STRING Isolation details (such as missing field names)

Isolation reason codes

error_code error_key Condition
100 JSON_PARSE_ERROR JSON parsing failed
200 DATETIME_MISSING The dateTime field is missing or empty
201 DATETIME_INVALID dateTime cannot be parsed
— DATASOURCE_MISSING _dataSource is null or an undefined value

Isolated log query example

You can query isolated logs directly in BigQuery. To query them in BigQuery, you must first request BigQuery access.

SELECT
  errorReason,
  errorStackTrace,
  eventName,
  dateTime,
  rawData,
  bigqueryRegistTimestamp
FROM `habq-storage.analytics_N_live.t_raw_event_log_rescue`
WHERE appIdGroup = 'com.sample.test'  -- your actual appIdGroup
LIMIT 100;

Learn more