> For the complete documentation index, see [llms.txt](https://liuyang89116.gitbook.io/system-design/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://liuyang89116.gitbook.io/system-design/chapter-2/metrics_aggregation/ad_logging.md).

# Design Ads Logging System

## Design Ads Logging System

## Requirements

### Functional requirements

* Generic ads logging system
* Advertiser - provide ads - want users to convert
* Publisher - display ads
* Influence - ranking of the ads
* Log publisher page information, end user information
* calculate the count of different ads(click, display, etc.)
* Ad Event: display, click, conversion

### Non-functional requirements

* Scalability
* Lower latency
* High availability

***

## High-level Architecture

![](https://2407442552-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Lpv9LvHzpublmUWisvz%2Fuploads%2Fgit-blob-e6045e0894b9b22f034d01ee8217da07978b77f0%2Fads_logging.png?alt=media)

***

## Detailed Design

### Ads Publish Service

* user can publish new ads
* new ads request will send to Ads Service
* store ads metadata in the SQL database

Ads Table

| adId(PK) | userID | description | created\_time |
| -------- | ------ | ----------- | ------------- |
| 123485   | 4541   | "new ad"    | 456487        |

***

### Ads Service

* rank ads
* other ads related operation

#### Search/Ads Ranking

* ads relevance check
* impression count
* CTR: Click-Through Rate
* CVR: Conversion Rate

***

### Logging Service

* we can use lambda architecture to do handle calculation of how many counts for different ads
* Real-time processor is the speed layer - not that accurate count (calculate lastest 1 min, 5 min results)
* Batch Processor is the batch layer - accurate count (calculate recent 1 day, 1 month, 1 year results)

#### Message Request

```json
{
    eventId: 1254, // specify a unique ad
    advertiserId: 15464,
    userId: 676467,
    eventType: "click",
    timestamp: 45613854,
}
```

#### Real-time Processor

* it contains **multiple consumers** to poll messages from the queue
* impression calculated by the stream processor; impression + 1 if it is the user
* sharding based on `eventId` or `advertiserId`
* each of the consumer can handle all events for the same eventId
* it is write-heavy, we can use Cassandra (append only) or ElastiSearch to store the results
* in the processor, we can also build in-memory hashtable
  * use **LSM(log-structured merge-tree)** as the data structure

#### Batch Processor

* log all the data
* map-reduce job for accurate data
* mainly for data correction and monitoring