Getting started
At the time of writing, the overall system is not yet production-ready. However, if you want to check out what the system is about, you can follow these instructions to set up a local development system that runs on your own infrastructure. Of course, for most deep learning training or prediction tasks you will require a NVIDIA GPU.
The server backend and worker nodes are expected to run Linux (tested with Ubuntu 18.04). The frontend, e.g., when using ADAMS can be Linux, Windows or Mac.
Set up the backend
Clone the following repositories (within the same directory):
git clone https://github.com/waikato-ufdl/ufdl-backend git clone https://github.com/waikato-ufdl/ufdl-json-messages
Change into the directory of the cloned ufdl-backend repository and run the following script to set up the virtual environment for the server (CAUTION: it will delete any previously stored data and the database):
./dev_init.sh
Once this has completed, you can start up the REST API on 127.0.0.1
as follows:
./dev_start.sh
Use 0.0.0.0:8000
as argument if you want to make the server available to the outside world on port 8000.
Ensure that your firewall allows that port to be accessed from the outside.
Initialize the backend
Download the ZIP file of the ADAMS frontend and unzip it.
Start ADAMS with the
bin/start_gui.sh
script (Linux/Mac) orbin/start_gui.bat
batch file (Windows).Use the Flow editor (from the Tools menu) to run the
adams-ufdl-all-basic_setup.flow
flow for setting up a basic environment (users, teams, projects).
Set up a worker node
On the worker node, clone the following repositories (within the same directory):
git clone https://github.com/waikato-ufdl/ufdl-json-messages git clone https://github.com/waikato-ufdl/ufdl-python-client git clone https://github.com/waikato-ufdl/ufdl-job-launcher
Change into the directory of the cloned ufdl-job-launcher repository and run the following script to set up the virtual environment:
./dev_init.sh
In the examples
directory, you can copy the job-launcher-example.conf
configuration to job-launcher.conf
and then update the required parameters (if anything, should be only the url
).
Once this suits your system, you can start the job-launcher like this (from within the ufdl-job-launcher
directory):
./venv.dev/bin/ufdl-joblauncher -C examples/job-launcher.conf -C
Use the system
The following ADAMS flows are available to manage your datasets and run jobs (simply execute them with the Flow editor):
adams-ufdl-core-manage_backend.flow
- for managing the backend, starting jobs, etc.adams-ufdl-image-manage_image_classification_datasets.flow
- manages image classification datasetsadams-ufdl-image-manage_objected_detection_datasets.flow
- manages object detection datasetsadams-ufdl-speech-manage_speech_datasets.flow
- manages speech datasets