
## Talo + ClickHouse = 🚄

Sadly, there's no emoji that can accurately convey the speed and scalability improvements we've made to the Events service. We've updated the service to use ClickHouse, a column-oriented database management system that's designed for real-time analytics.

This migration will provide better performance and scalability for the Events service, ensuring that it can handle the increasing volume of events generated by your games and providing you with quick access to your players' event data.

### Taking a step back: what are events?

Events are an easy and declarative way of measuring player behaviour in your game. You can use events to track player actions, such as logging in, completing a level, or purchasing an item. By tracking these events, you can gain insights into how players are interacting with your game and use this data to improve your game's design and player experience.

Talo lets you aggregate events, filter them and drill down into specific events on an individual player level. Events are incredibly powerful for debugging too - you can follow a player's journey through their events to understand potential issues they might be facing.

You can learn more about Talo's [Events service here](https://trytalo.com/events).

### Why did we migrate to ClickHouse?

ClickHouse is optimised for running analytical queries on large datasets, making it perfect for the Events service which needs to handle a high volume of events generated by your games. It also provides lightning fast access to your event data, allowing you to quickly query and analyse your players' behaviour.

Previously, we noticed large dips in performance when fetching large sets of events (for example when the event date range was upwards of a year). This impacted both the [homepage of the dashboard](https://dashboard.trytalo.com) where we show a summary of events, the [Events overview](https://trytalo.com/events) where you can visualise events and the player events page where you can drill down into specific player journeys.

We want Talo to be as scalable as possible and ClickHouse is a key part of that vision. The migration to ClickHouse will ensure that Talo can handle events from games of any size - whether you're building a small indie game or a large-scale multiplayer project.

### How did we do it?

To use ClickHouse, we had to drop the relational way of storing events and move to a column-oriented database. This means that we store data in columns rather than rows, which allows for faster read and write operations.

We migrated our existing `events` table into two tables: one for `events` and one for `event_props` (the metadata associated with each event). This separation allows us to store the event data in a more efficient way and makes it easier to query and analyse the data. We don't always need to fetch the props so separating them out allows us to be more efficient with our queries.

We also made sure the new tables were atomic meaning that they didn't rely on external data that exists in the operational database. This ensures that the Events service is resilient to operational database changes and can continue to function independently.

After updating our other services (like the player and dashboard headlines services) to use the new ClickHouse driver, we were ready to start migrating data to the new ClickHouse database.

To do this, we ran SQL queries to pull out the existing data into the new format as CSVs (one for `events` and one for `event_props`) and inserted that directly into the ClickHouse database using the CSVFormat. Whether you have JSON to insert, a CSV file or even an SQL query, ClickHouse is incredibly flexible when it comes to inserting data.

Talo is completely open-source, so you can check out the [migration PR here](https://github.com/TaloDev/backend/pull/324) if you're interested.

## Leaderboard and resilience updates

We've also been hard at work making improvements to our [Godot plugin](https://trytalo.com/godot) and [Unity package](https://trytalo.com/unity). We recently introduced leaderboard entry caches to reduce the number of requests made to the Talo API when fetching leaderboard entries. This should make it easier to build leaderboards and improve the performance and responsiveness of leaderboards in your game.

You can check out the docs for the [Godot plugin entry cache here](https://docs.trytalo.com/docs/godot/leaderboards#entry-cache) and the [Unity package entry cache here](https://docs.trytalo.com/docs/unity/leaderboards#entry-cache).

Our focus is currently on making Talo as resilient as possible. To do this, we're looking into adding more local caching functionality for handling network loss/drop-out issues. For example, if Talo is down for maintenance in the future, or a player's network temporarily drops out, we want to ensure that your game can continue to function as expected and that any changes made are reflected in Talo after a connection is re-established.

## Join the community

Looking to use Talo? You can find the Talo community on [Discord](https://discord.gg/2RWwxXVY3v). Come for a chat, ask questions or share your feedback with us. We'd love to hear from you!


Talo’s Events service now uses ClickHouse for better performance and scalability.

Events just got faster: migrating to ClickHouse


## Quick and easy leaderboards

Leaderboards are a tried-and-tested way of keeping players engaged with your game. They can be used to show off the best players, encourage competition or even just to give players a sense of progression. In this post, we'll go over how to create a simple leaderboard system in Godot using the Talo Godot plugin.

Here's an example of what we'll be building:

![The leaderboard scene displaying multiple entries](/blog/leaderboards-godot/leaderboard-scene.png)

## Leaderboards through Talo

Talo makes it easy to create leaderboards and interact with them. You can even view real time leaderboard updates in the Talo dashboard. The leaderboards service are [customisable](https://trytalo.com/leaderboards) and are designed to work across many different genres of games.

Using the Talo Godot plugin, you can easily fetch entries from the leaderboards service without having to set up your own database, webserver or API. Best of all, you can remove suspicious entries directly from the dashboard, making sure your leaderboards are fair and accurate.

Leaderboards in Talo can also be synced up with Steamworks leaderboards, taking advantage of Steam's ecosystem and Talo's observability.

![A leaderboard's entries displayed in the Talo dashboard](/blog/leaderboards-godot/leaderboard-entries.png)

## Setup the Talo Godot plugin

Before we start building our leaderboards, we need to set up the Talo Godot plugin. You can do this by downloading the plugin from the [Godot Asset Library](https://godotengine.org/asset-library/asset/2936) (or by searching "Talo Game Services" in the `AssetLib` tab of the Godot editor). You can also find the latest version of the plugin on [itch.io](https://sleepystudios.itch.io/talo-godot).

Once you've downloaded the plugin, import it (typically you want to install plugins under the `addons` folder) and enable it in the `Project > Project Settings > Plugins` window.

![Install Talo from the AssetLib tab](/blog/leaderboards-godot/install-assetlib.png)

You can now run your game - this will create a `settings.cfg` asset inside the folder where you installed the plugin - we'll be using this later. To learn more about how to configure Talo, including how to configure your self-hosted version, [check out the docs](https://docs.trytalo.com/docs/godot/install#create-and-update-settings).

![Enable the Talo plugin in the project settings](/blog/leaderboards-godot/install-enable.png)

### Generating an access key

Next we need to generate an access key for your game. Visit the [Talo dashboard](https://dashboard.trytalo.com), login or create an account (and confirm your email address). Once logged in, create a new game (if you need to) and visit the Access Keys page.

Once there, choose the scopes available to your access key. For this tutorial, you'll need the `read:players`, `write:players`, `read:leaderboards` and `write:leaderboards` scopes. Create your access key and copy it into the `access_key` field of the `settings.cfg`.

![Creating a new access key in the Talo dashboard](/blog/leaderboards-godot/access-key.png)

### Creating a leaderboard

The last bit of setup we need to do involves actually creating our leaderboard. Visit the [Leaderboards page](https://dashboard.trytalo.com/leaderboards) in the dashboard. You can also access it from the `Services` dropdown. Configure your leaderboard however you like but keep note of your chosen `Internal name` since you'll need this to fetch entries.

![Creating a new leaderboard in the Talo dashboard](/blog/leaderboards-godot/dashboard-leaderboard.png)

## Scene setup

We only need two scenes for this tutorial - the leaderboard entry scene and the leaderboard scene. The entry scene will display leaderboard entry data and the leaderboard scene will display all the leaderboard entries and allow us to submit new ones.

### The leaderboard entry scene

We'll be using a simple scene with a single `Label` node to display entry data which includes the position, player identifier and score of a single entry.

Start off by creating a new scene where the root node is a `Label`. Next, update the text of the label to match the following:

```
{pos}. {username} scored: {score} points
```

We want to template out the text in the label using a script. Attach a new script to your root node with the following content:

```gdscript
extends Label

func _set_pos(pos: int) -> void:
  text = text.replace("{pos}", str(pos + 1))

func _set_username(username: String) -> void:
  text = text.replace("{username}", username)

func _set_score(score: int) -> void:
  text = text.replace("{score}", str(int(score)))

func set_data(pos: int, username: String, score: int) -> void:
  _set_pos(pos)
  _set_username(username)
  _set_score(score)
```

This script will allow us to set the position, username and score of the entry. When the label is fully populated we should see something like this:

```
1. Boz scored: 300 points
2. Baz scored: 200 points
3. Biz scored: 100 points
```

We'll call the `set_data()` function from the main leaderboard script.

### The leaderboard scene

The leaderboard scene will be a bit more complex. We'll need a number of things to build out this simple leaderboard UI:

1. A `ScrollContainer` with a `VBoxContainer` child which will act as the parent node for our leaderboard entries.
2. A `TextEdit` node which will allow us to input a new username.
3. A `Button` node which will allow us to submit a new entry to the leaderboard.

Some other optional nodes you might want to add include a `Label` node to display information about new leaderboard updates and a `Label` to display the name of the leaderboard. Here's the structure we're using:

![The leaderboard scene structure](/blog/leaderboards-godot/leaderboard-scene-structure.png)

The `%` symbol here refers to a unique node name. You can do this by right-clicking the node and choosing the `Access as Unique Name` option. It's important to make the node names unique so we can easily reference them in our script.

We've also connected the `on_pressed` signal into a script attached to the `Leaderboard` root node.

The [Talo Godot plugin](https://godotengine.org/asset-library/asset/2936) comes with a leaderboards scene sample in the `samples/leaderboards` folder you can use as a basis for your own scene (this is what we're building!)

## Fetching entries

Now that we have our scenes set up, we can start fetching entries from the leaderboard. We'll be using the `Talo.Leaderboards.get_entries()` function to do this. This function takes a `leaderboard_name` and a `page` parameter. The `leaderboard_name` is the internal name of the leaderboard we created earlier and the `page` is the current page we want to fetch. Requests are paginated to make sure we're not fetching too much data at once.

Here's the starting point for our leaderboard script:

```gdscript
extends Node2D

var entry_scene = preload("res://addons/talo/samples/leaderboards/entry.tscn") # update this to a path that matches your entry scene

@export var leaderboard_internal_name: String

@onready var entries_container: VBoxContainer = %Entries
@onready var username: TextEdit = %Username

func _ready() -> void:
  await _load_entries()

func _create_entry(entry: TaloLeaderboardEntry) -> void:
  var entry_instance = entry_scene.instantiate()
  entry_instance.set_data(entry.position, entry.player_alias.identifier, entry.score)
  entries_container.add_child(entry_instance)

func _build_entries() -> void:
  for child in entries_container.get_children():
    child.queue_free()

  for entry in Talo.leaderboards.get_cached_entries(leaderboard_internal_name):
    _create_entry(entry)

func _load_entries() -> void:
  var page = 0
  var done = false

  while !done:
    var res = await Talo.leaderboards.get_entries(leaderboard_internal_name, page)
    var entries = res[0]
    var last_page = res[2]

    if last_page:
      done = true
    else:
      page += 1

  _build_entries()

func _on_submit_pressed() -> void:
  pass
```

Lets break this down piece by piece:

1. We preload the entry scene so we can instantiate it later.
2. We define an `export` variable `leaderboard_internal_name` which we'll use to set the internal name of the leaderboard we want to fetch entries from. You will need to remember to set this in the Godot editor to the `Internal name` of the leaderboard you created earlier in the dashboard.
3. We define two `onready` variables - `entries_container` which is the `VBoxContainer` we'll be adding our leaderboard entries to and `username` which is the `TextEdit` node we'll be using to identify players.
4. In the `_ready()` function we call `_load_entries()` which will fetch entries from the leaderboard.
5. The `_create_entry()` function instantiates a new entry scene and sets the data for the entry.
6. `_load_entries()` and `_build_entries()` are used to fetch leaderboard entries and construct the UI respectively. 

Diving deeper into the `_load_entries()` function we can see that it looks through `Talo.leaderboards.get_entries()` responses until it reaches the last page. `get_entries()` returns an array of the following:

1. The entries
2. The total number of entries available
3. If we're on the last page

After we've fetched entries from the last page, the function calls `_build_entries()`.

`_build_entries()` is what we use to keep our leaderboard UI up-to-date. It first clears the previous entries state and then loops through the cached entries (made up of every entry that Talo fetches for a specific leaderboard) and attaches a child entry scene for each one.

## Submitting entries

You'll notice that we've left the `_on_submit_pressed()` function empty. This is where we'll be submitting new entries to the leaderboard. We'll be using the `Talo.Leaderboards.add_entry()` function to do this. This function takes the `leaderboard_name` and the `score` of the entry we want to submit.

Before we can submit an entry, we need to `identify()` our player. This allows us to tie an entry back to a specific player. We can do this by calling `Talo.Players.identify()` with the `service` and `identifier` of the player. The `service` is the identity service we're using (in this case `username`) and the `identifier` is the unique identifier of the player which we'll get from the `username` field:

```gdscript
func _on_submit_pressed() -> void:
  await Talo.players.identify("username", username.text)

  var score = RandomNumberGenerator.new().randi_range(0, 100)
  var res = await Talo.leaderboards.add_entry(leaderboard_internal_name, score)

  _build_entries()
```

In summary this function is doing the following:
1. Identifying the player using the `username` value.
2. Generating a random score between 0 and 100.
3. Submitting the entry to the leaderboard.
4. Rebuilding the entries UI.

The new entry is automatically added to the entries cache which means we don't need to fetch the leaderboard entries again and we can just call `_build_entries()` to update the UI with the latest dataset.

## Testing it out

To verify that everything is working as expected, run your game and check the leaderboard scene. You should see the entries displayed in the order you selected when you created the leaderboard in the dashboard. You can also submit new entries and see them appear in the leaderboard. Here's the leaderboard we built:

![The leaderboard scene displaying multiple entries](/blog/leaderboards-godot/leaderboard-scene.png)

We've added a few bells and whistles to ours. Remember, you can find this example in the Talo Godot plugin in the `samples/leaderboards` folder (along with the entry scene and all the scripts).

## Wrapping up

In only a few minutes you can have a fully functioning leaderboard into your Godot game. We've covered how to set up the [Talo Godot plugin](https://trytalo.com/godot), create a leaderboard in the Talo dashboard, fetch entries from the leaderboard and submit new entries. There's plenty more you can do from here such as adding pagination, filtering entries by player or even adding a leaderboard for each level in your game.

We're constantly improving Talo, including the leaderboards system which recently received an update to add the caching mechanism. If you have any feedback or want to receive updates on new features, come and join our [Discord community](https://discord.gg/2RWwxXVY3v) or follow us on [GitHub](https://github.com/TaloDev).


An end-to-end example of how to build a simple, scalable and interactive leaderboard into your Godot game.

How to quickly and easily create leaderboards in Godot


## Authentication audit logs are here

We've added a new layer of security to Talo's recently released Authentication service: audit logs designed for keeping track of player authentication actions. With the new audit logs, you can monitor all player authentication actions, including registrations, logins, password resets, and more.

### What's available in the audit logs?

The following actions are now recorded in the audit logs for individual players:
- **Registering**: When a player creates a new account.
- **Logging in**: When a player successfully logs in.
- **Logging out**: When a player logs out.
- **Verification started**: When a player starts the verification process.
- **Verification failed**: When a player fails the verification process.
- **Password changed**: When a player changes their password.
- **Password change failed**: When a player fails to change their password.
- **Email changed**: When a player changes their email address.
- **Email change failed**: When a player fails to change their email address.
- **Password reset requested**: When a player requests a password reset.
- **Password reset completed**: When a player successfully resets their password.

The logs also provide extra details like IP address and user agent to provide additional context for each action. Contextual details are provided for specific actions: for example, when a player changes their email address we'll log the old email address.

### Where can I see the audit log?

You can access the audit logs directly from the Talo dashboard. Simply navigate to the `Players` service, select a player, and their audit logs will be available on their profile if they are using Talo Player Authentication.

![Audit logs in the Talo dashboard](/blog/authentication-audit-logs/auth-activity.png)

## New API: toggling verification

We've snuck in a small update to Authentication too!

Previously, players could only enable verification on registration, and now we've added an API to enable verification for existing players. This allows you to require verification for all players, even if they've already registered, or allow players to decide to enhance the security of their account.

Players can also choose to disable verification entirely. Players will need to supply their current password to enable or disable verification.

The new API is now available in our Unity package (v0.20.0) and Godot plugin (v0.4.0), and the existing player authentication documentation has been updated to show how to use this new API.

### Feedback and suggestions

We're keen to hear what other items would be useful to include in the audit logs. If you have any feedback or suggestions, please reach out to us on our [Discord](https://discord.gg/2RWwxXVY3v) or raise an issue in our [GitHub repository](https://github.com/talodev/backend).


Keep track of player authentication actions with Talo’s new audit logs.

Authentication audit logs now available

Continuity is a new resilience layer for keeping your game data in sync.

Sign up

Introducing Talo Continuity

Continuity is here!

How does it work?

Configuring Continuity

Now available in the Talo Godot plugin

Join our community

Get started

More from the Talo Blog

Events just got faster: migrating to ClickHouse

How to quickly and easily create leaderboards in Godot

Authentication audit logs now available