Skip to content

Event Storage Structure

Overview

You can see how transmitted event logs are stored in BigQuery, the detailed structure of the attributes JSON field, and how logs that violate transmission rules are quarantined.

Table Structure

Sent event logs are stored in the t_raw_event_log table in the company-specific BigQuery dataset.

Table Column Structure

Storage Location

Item Format
Table t_raw_event_log
Partitioning Daily partition by dateTime

Column Definition

Column name Data type Required Description Example
userId STRING Y Account-based user identifier 100001
identifierProvider STRING Y System that issues the user identifier. Always hive hive
deviceId STRING Y Device-based user identifier 100000
appId STRING Y Project ID registered in Hive Console > App Center "com.com2us.game.ios"
appIdGroup STRING Y Game ID issued by Hive Console > App Center. Value converted based on appId com.com2us.game
dateTime TIMESTAMP Y Time when the event occurred. eventTime converted to UTC 2026-07-20T05:01:01Z
eventName STRING Y Event name that classifies the event asset_drop
checksum STRING N Hash value generated only from the original data sent by the customer, excluding all fields injected by the pipeline or generated by the system (such as hiveAttributes) HW2FTEB56TEEWER
bigqueryRegistTimestamp TIMESTAMP N The UTC time when the event was stored in BigQuery 2026-07-20T05:02:01Z
attributes JSON N JSON field where custom and auto-collected attributes are stored See below

Table Structure Precautions & Tips

  • The same event may be stored redundantly due to retransmission, etc., so it is recommended to use a duplicate removal query based on checksum when searching.

    QUALIFY ROW_NUMBER() OVER (PARTITION BY checksum ORDER BY bigqueryRegistTimestamp DESC) = 1

  • Event logs that violate transport rules are stored in the rescue table, not this table. For more information, see Data Isolation.

attributes Detailed Attributes

Note

The attributes column is of type JSON and is divided into two areas: hiveAttributes and eventAttributes.

attributes Example

{
  "hiveAttributes": {
    "_clientIp": "220.70.82.0",
    "_dataSource": "custom_server",
    "_fluentd_appid": "com.com2us.game.ios",
    "_fluentd_sender_ip": "43.101.101.33",
    "_fluentd_hostaname": "9104813d31ab",
    "_fluentd_tag": "ha2union.game.stage",
    "_geoIpCountry": "KR",
    "_market": "google",
    "_os": "android"
  },
  "eventAttributes": {
    "eventTime": "2026-07-20T14:01:01+09:00",
    "level": 10,
    "character_name": "AA",
    "stage_id": "stage_101"
  }
}

hiveAttributes

This is the area that the pipeline automatically processes and stores.

Attribute name Data type Description Example
_clientIp STRING If client_ip or clientIp is sent with the log, the sent IP with its last octet changed to 0. Not created if client_ip or clientIp is not sent 220.70.82.0
_geoIpCountry STRING Country code determined by the pipeline based on client_ip or clientIp (ISO 3166 alpha-2). Not created if client_ip or clientIp is not sent KR
_dataSource STRING Path through which the event log actually came in, as determined by the pipeline. See the _dataSource details below custom_server
_fluentd_appid STRING appId of the log collector (Fluentd) as determined by the pipeline com.com2us.game.ios
_fluentd_sender_ip STRING IP of the log collector (Fluentd) as determined by the pipeline 43.101.101.33
_fluentd_hostaname STRING Hostname of the log collector (Fluentd) as determined by the pipeline 9104813d31ab
_fluentd_tag STRING Name of the tag sent to the log collector (Fluentd) as determined by the pipeline ha2union.game.stage
_market STRING Market information determined by Analytics from the appId google
_os STRING OS information determined by Analytics from the appId android

_dataSource Details

_dataSource indicates the path through which the event log actually came in, and is automatically set to one of the following values:

Value Description
airbridge Events collected via Airbridge
appsflyer Events collected via Appsflyer
adjust Events collected via Adjust
singular Events collected via Singular
hive_server Events automatically collected through Hive authentication, billing, etc.
hive_sdk Events such as funnels automatically collected by Hive SDK
custom_server Custom events sent by the customer from the server
custom_sdk Custom events sent by customers via the event log submission function of Hive SDK
custom_sdk_mig Custom events that customers send to older versions of Analytics via the Send Event Log feature in Hive SDK
custom_server_mig Custom events that customers send from their servers to older versions of Analytics
custom_except An event that generates and transmits eventAttribute, hiveAttribute as an exceptional case during the event conversion process

eventAttributes

This is the area where the attributes sent by the user and the attributes automatically collected by SDK are stored.

Attribute name Data type Description
eventTime STRING eventTime original value sent as required attribute
(User Defined Attribute) numbers/characters Custom attributes sent directly by users
(auto-collected attributes) Varies by attribute Client information automatically collected by SDK (when transmitting client)

attributes Precautions & Tips

  • hiveAttributes is not an area where the developer sets the value directly, but an area that the pipeline automatically fills. Values other than _clientIp, _dataSource, _fluentd_appid, _fluentd_sender_ip, _fluentd_hostaname, _fluentd_tag, _geoIpCountry, _market, and _os are not saved.
  • Both custom attributes and SDK auto-collected attributes are stored together in eventAttributes. When searching, they must be distinguished by attribute name.
  • You can search by distinguishing the transmission path of the event log (SDK/server/MMP, etc.) using the _dataSource value.

Data Isolation

Warning

Event logs that do not comply with the transport rules are not stored in the normal table (t_raw_event_log), but are stored separately in the quarantine table (t_raw_event_log_rescue). The quarantine table preserves the original JSON, so you can determine the cause of the quarantine, fix it, and retransmit.

Reason for Quarantine

Error reason Description How to respond
Required attribute missing Any of the 7 required attributes (userId, identifierProvider, deviceId, appId, appIdGroup, eventTime, eventName) is missing or has an empty value Resend with missing required attributes
Use of reserved words When attributes, eventAttributes, and hiveAttributes are used as attribute names (case-insensitive) Change the attribute name to another name
Attribute value not allowed If the attribute value is in the form of a JSON object ({}) or a JSON array ([]) Change attribute value to string or number
eventTime format error eventTime is not in RFC 3339 format or cannot be parsed Modify to RFC 3339 format (including time zone)
eventTime out of range eventTime is outside the range of 10 years in the past to 5 years in the future as of the current time Check the eventTime value and modify it to the normal range
JSON parsing failed The sent message is not in a valid JSON format Verify the JSON format and resend

Isolated Table Structure

Isolated event logs are stored in the t_raw_event_log_rescue table with the following structure. Each column is filled only when the original JSON can be parsed; otherwise, it is stored as an empty value. You can check the original in rawData.

Column name Data type Description
userId STRING Sent userId
identifierProvider STRING Sent identifierProvider
deviceId STRING Sent deviceId
appIdGroup STRING Sent appIdGroup
dateTime STRING Original value of transmitted eventTime (Save as string, not TIMESTAMP)
eventName STRING Sent eventName
bigqueryRegistTimestamp TIMESTAMP Time when the log was stored in BigQuery
checksum STRING Hash value generated only from the original data sent by the customer, excluding all fields injected by the pipeline or generated by the system (such as hiveAttributes)
rawData STRING All original JSON sent
errorReason STRING Isolation reason code. See Isolation Reason Codes
errorStackTrace STRING Quarantine details (missing field names, etc.)

Isolation Reason Codes

error_code error_key Condition
100 JSON_PARSE_ERROR JSON parsing failed
200 DATETIME_MISSING dateTime field is missing or empty
201 DATETIME_INVALID dateTime cannot be parsed
— DATASOURCE_MISSING _dataSource is null or an undefined value

Isolated Log Query Example

You can query isolated logs directly in BigQuery. To query with BigQuery, you must first request BigQuery permissions.

SELECT
  errorReason,
  errorStackTrace,
  eventName,
  dateTime,
  rawData,
  bigqueryRegistTimestamp
FROM `habq-storage.analytics_N_live.t_raw_event_log_rescue`
WHERE appIdGroup = 'com.sample.test'  -- your actual appIdGroup
LIMIT 100;