commit	436275148cf8ef4d116c7e4dd040cbba220a7fc7	[log] [tgz]
author	Rafael Novello <rafa.reis.novello@gmail.com>	Tue Jul 03 03:29:24 2018 -0300
committer	Daniel Martins Takabayashi <daniel.takabayashi@gmail.com>	Mon Jul 02 23:29:24 2018 -0700
tree	f790c274839b15910ad5a97d9068f2c0731689e5
parent	ebb2bb39a1fdcc5bd564fbd637c6fafb31673186 [diff]

tree: f790c274839b15910ad5a97d9068f2c0731689e5

README.md

Marvin Toolbox v0.0.3

Quick Start

Review

Marvin is an open-source Artificial Intelligence platform that focuses on helping data scientists deliver meaningful solutions to complex problems. Supported by a standardized large-scale, language-agnostic architecture, Marvin simplifies the process of exploration and modeling.

Getting Started

Installing Marvin (Ubuntu and MacOS user)
Installing Marvin (Other OS)
Creating a new engine
Working in an existing engine
Command line interface
Running an example engine

Installing Marvin as Ubuntu and MacOS user

Perform the following steps to install the Marvin Toolbox:

Libsasl2-dev, Python-pip and Graphviz installation

Ubuntu: 
sudo apt-get install libsasl2-dev python-pip graphviz -y

MacOS: 
sudo easy_install pip
brew install openssl graphviz

VirtualEnvWrapper Installation

sudo pip install --upgrade pip
sudo pip install virtualenvwrapper --ignore-installed six

Spark installation

curl https://d3kbcqa49mib13.cloudfront.net/spark-2.1.1-bin-hadoop2.6.tgz -o /tmp/spark-2.1.1-bin-hadoop2.6.tgz

sudo tar -xf /tmp/spark-2.1.1-bin-hadoop2.6.tgz -C /opt/
sudo ln -s /opt/spark-2.1.1-bin-hadoop2.6 /opt/spark

echo "export SPARK_HOME=/opt/spark" >> $HOME/.bash_profile

If there is no /opt directory present, before unpacking spark, run:

sudo mkdir /opt

Set environment variables

echo "export WORKON_HOME=$HOME/.virtualenvs" >> $HOME/.bash_profile
echo "export MARVIN_HOME=$HOME/marvin" >> $HOME/.bash_profile
echo "export MARVIN_DATA_PATH=$HOME/marvin/data" >> $HOME/.bash_profile
echo "source virtualenvwrapper.sh" >> $HOME/.bash_profile

source ~/.bash_profile

Clone and install python-toolbox

mkdir $MARVIN_HOME
mkdir $MARVIN_DATA_PATH
cd $MARVIN_HOME

git clone https://github.com/marvin-ai/marvin-python-toolbox.git
cd marvin-python-toolbox

mkvirtualenv python-toolbox-env
setvirtualenvproject

make marvin

Test the installation

marvin test

Installing Marvin with Other OS

Perform the following steps to install Marvin Toolbox using Vagrant:

Install requirements

Virtual box (Version 5.1 +)
Vagrant (Version 1.9.2 or +)

Clone the repository and start provision

git clone https://github.com/marvin-ai/marvin-vagrant-dev.git
cd marvin-vagrant-dev

Prepare the dev (engine creation) box

vagrant up dev
vagrant ssh dev

Wait for provision process and follow interactive configuration script after access the dev box using vagrant ssh command.

The marvin source projects will be located in your home folder; to compile and use the Marvin toolbox

workon python-toolbox-env
make marvin

Creating a new engine

To create a new engine

workon python-toolbox-env
marvin engine-generate

Respond to the prompt and wait for the engine environment preparation to complete. Don't forget to start dev box before if you are using vagrant.

Test the new engine

workon <new_engine_name>-env
marvin test

For more information

marvin --help

Working in an existing engine

Set VirtualEnv and get to the engine's path

workon <engine_name>-env

Test your engine

marvin test

Bring up the notebook and access it from your browser

marvin notebook

Command line interface

Usage: marvin [OPTIONS] COMMAND [ARGS]

Options:

  --debug       #Enable debug mode.
  --version     #Show the version and exit.
  --help        #Show this command line interface and exit.

Commands:

  engine-generate     #Generate a new marvin engine project.
  engine-generateenv  #Generate a new marvin engine environment.
  engine-grpcserver   #Marvin gRPC engine action server starts.
  engine-httpserver   #Marvin http api server starts.
  hive-dataimport     #Import data samples from a hive databse to the hive running in this toolbox.
  hive-generateconf   #Generate default configuration file.
  hive-resetremote    #Drop all remote tables from informed engine on host.
  notebook            #Start the Jupyter notebook server.
  pkg-bumpversion     #Bump the package version.
  pkg-createtag       #Create git tag using the package version.
  pkg-showchanges     #Show the package changelog.
  pkg-showinfo        #Show information about the package.
  pkg-showversion     #Show the package version.
  pkg-updatedeps      #Update requirements.txt.
  test                #Run tests.
  test-checkpep8      #Check python code style.
  test-tdd            #Watch for changes to run tests automatically.
  test-tox            #Run tests using a new virtualenv.

Running a example engine

Clone the example engine from the repository

git clone https://github.com/marvin-ai/engines.git

Generate a new Marvin engine environment for the Iris species engine

workon python-toolbox-env
marvin engine-generateenv ../engines/iris-species-engine/

Run the Iris species engine

workon iris-species-engine-env
marvin engine-dryrun

Marvin is a project started at B2W Digital offices and released open source on September 2017.