commit | d01f04bbd3c1e5471fe4919acb77da5112a711f6 | [log] [tgz] |
---|---|---|
author | Rafael Novello <rafa.reis.novello@gmail.com> | Fri Jul 20 12:11:42 2018 -0300 |
committer | Daniel Martins Takabayashi <daniel.takabayashi@gmail.com> | Fri Jul 20 12:11:42 2018 -0300 |
tree | ed14d454402e055c0547727a312453576b72c0e5 | |
parent | 6e6b3923750036e7cdcec2b735003bd032c751eb [diff] |
Reproducing toolbox setup.py changes at engine template. Each engine … (#105) * Reproducing toolbox setup.py changes at engine template. Each engine will install its own dependencies. close #104 * some adjustments in toolbox and template makefiles #104 * Removing some commands by install mode (dev and prod) #104 * Moving autocomplete and notebook extension from toolbox setup to engine template setup. Close #107
Marvin is an open-source Artificial Intelligence platform that focuses on helping data scientists deliver meaningful solutions to complex problems. Supported by a standardized large-scale, language-agnostic architecture, Marvin simplifies the process of exploration and modeling.
Perform the following steps to install the Marvin Toolbox:
Ubuntu: sudo apt-get install libsasl2-dev python-pip graphviz -y MacOS: sudo easy_install pip brew install openssl graphviz
sudo pip install --upgrade pip sudo pip install virtualenvwrapper --ignore-installed six
curl https://d3kbcqa49mib13.cloudfront.net/spark-2.1.1-bin-hadoop2.6.tgz -o /tmp/spark-2.1.1-bin-hadoop2.6.tgz sudo tar -xf /tmp/spark-2.1.1-bin-hadoop2.6.tgz -C /opt/ sudo ln -s /opt/spark-2.1.1-bin-hadoop2.6 /opt/spark echo "export SPARK_HOME=/opt/spark" >> $HOME/.bash_profile
If there is no /opt directory present, before unpacking spark, run:
sudo mkdir /opt
echo "export WORKON_HOME=$HOME/.virtualenvs" >> $HOME/.bash_profile echo "export MARVIN_HOME=$HOME/marvin" >> $HOME/.bash_profile echo "export MARVIN_DATA_PATH=$HOME/marvin/data" >> $HOME/.bash_profile echo "source virtualenvwrapper.sh" >> $HOME/.bash_profile source ~/.bash_profile
mkdir $MARVIN_HOME mkdir $MARVIN_DATA_PATH cd $MARVIN_HOME git clone https://github.com/marvin-ai/marvin-python-toolbox.git cd marvin-python-toolbox mkvirtualenv python-toolbox-env setvirtualenvproject make marvin
marvin test
Perform the following steps to install Marvin Toolbox using Vagrant:
git clone https://github.com/marvin-ai/marvin-vagrant-dev.git cd marvin-vagrant-dev
vagrant up dev vagrant ssh dev
Wait for provision process and follow interactive configuration script after access the dev box using vagrant ssh command.
workon python-toolbox-env make marvin
workon python-toolbox-env marvin engine-generate
Respond to the prompt and wait for the engine environment preparation to complete. Don't forget to start dev box before if you are using vagrant.
workon <new_engine_name>-env marvin test
marvin --help
workon <engine_name>-env
marvin test
marvin notebook
Usage: marvin [OPTIONS] COMMAND [ARGS]
Options:
--debug #Enable debug mode. --version #Show the version and exit. --help #Show this command line interface and exit.
Commands:
engine-generate #Generate a new marvin engine project. engine-generateenv #Generate a new marvin engine environment. engine-grpcserver #Marvin gRPC engine action server starts. engine-httpserver #Marvin http api server starts. hive-dataimport #Import data samples from a hive databse to the hive running in this toolbox. hive-generateconf #Generate default configuration file. hive-resetremote #Drop all remote tables from informed engine on host. notebook #Start the Jupyter notebook server. pkg-bumpversion #Bump the package version. pkg-createtag #Create git tag using the package version. pkg-showchanges #Show the package changelog. pkg-showinfo #Show information about the package. pkg-showversion #Show the package version. pkg-updatedeps #Update requirements.txt. test #Run tests. test-checkpep8 #Check python code style. test-tdd #Watch for changes to run tests automatically. test-tox #Run tests using a new virtualenv.
git clone https://github.com/marvin-ai/engines.git
workon python-toolbox-env marvin engine-generateenv ../engines/iris-species-engine/
workon iris-species-engine-env marvin engine-dryrun
Marvin is a project started at B2W Digital offices and released open source on September 2017.