# Squirro Activity Log Plugin

Index activity logs from Squirro projects.

## Gather files

By default the plugin gathers the activity files from the path provided in the .ini configuration:

```
[activity]
path = /tmp/frontend
```

Custom path can be specified using the `source_folder` argument.

The plugin gathers the activity files from the root level and then check for files inside the folder with corresponding hostname.
If it finds a folder with the same hostname as the hostname that it is running on, it also searches for files inside this folder.

So assuming the dataloader runs on the instance with the hostname: `hostname-1`, it will gather the following files:

-   /tmp/frontend/activity.2022-01-01.jsonl
-   /tmp/frontend/hostname-1/activity.2022-01-02.jsonl

But won't gather the file below:

-   /tmp/frontend/hostname-2/activity.2022-01-03.jsonl

## Incremental Loading

The incremental loading is handled internally by the plugin using `now` column from the items.
The latest date is saved to the store and in the next plugin's run only newer items will be fetched.

## Loading Plugin

Using Squirro Toolbox:

```
squirro_asset -vvv dataloader_plugin upload -t $TOKEN --token ${CLUSTER} -f ../activity_plugin
```

where `$CLUSTER` can be for example `http://localhost:8300/` and token should be token generated for an admin user.
