Getting started

Install

The installation is quick and simple using pip.

# Anomaly Detection Framework needs a home directory to read settings from.
# In case is not provided, $HOME/anomdec will be used
export ANOMDEC_HOME=~/anomdec

# Install from pypi using pip
pip install anomalydetection

Sandbox

After installation, you can start a dashboard instance and play with the sandbox, there is no configuration required to do this.

# Start a dashboard to play with sandbox
anomdec dashboard

You will get an output similar to this one.

2018-07-09 12:37:51,930 - anomalydetection.anomdec.Anomdec:35 - INFO - Starting anomdec
2018-07-09 12:37:51,930 - anomalydetection.anomdec.Anomdec:38 - INFO - Run dashboard
2018-07-09 12:37:51,933 - anomalydetection.common.config.Config:50 - WARNING - Cannot load configuration.
[Errno 2] No such file or directory: '~/anomdec/anomdec.yml'

Note

Ignore the Config WARNING log if you are following this Getting started guide. This is because we have not configured any signal to process yet.

Devel mode

You can also run a devel mode. For this, you need to deploy a docker-compose.yml file. This will deploy an Apache Kafka, Apache Zookeeper and a Google PubSub emulator. The devel mode processes a random signals that are randomly generated by itself.

# Export kafka advertise hostname
export KAFKA_ADVERTISED_HOST_NAME=localhost

# Download docker-compose.yml file
curl -o docker-compose.yml https://raw.githubusercontent.com/bluekiri/anomalydetection/master/docker-compose.yml

# Start docker enviroment
docker-compose up

# In another terminal, start the devel mode
anomdec devel

You should see how messages are pushed to Kafka and PubSub backends to stdout.

- anomalydetection.anomdec.Anomdec:35 - INFO - Starting anomdec
- anomalydetection.anomdec.Anomdec:53 - INFO - Creating configuration for DEVEL MODE
- anomalydetection.anomdec.Anomdec:57 - INFO - Run dashboard, backend and producer
...
- anomalydetection.backend.stream.pubsub.PubSubStreamConsumer:70 - DEBUG - Message received: {"application": "devel0", "ts": "2018-07-09 13:00:29.886520", "value": 1}
- anomalydetection.backend.stream.pubsub.PubSubStreamConsumer:70 - DEBUG - Message received: {"application": "devel2", "ts": "2018-07-09 13:00:29.905861", "value": 3}
- anomalydetection.backend.stream.pubsub.PubSubStreamConsumer:70 - DEBUG - Message received: {"application": "devel1", "ts": "2018-07-09 13:00:29.902405", "value": 2}

Open the dashboard on your browser to visualize how data is processed in real time.