Quick Start

Smile is a fast and comprehensive machine learning system. With advanced data structures and algorithms, Smile delivers the state-of-art performance. Smile is self-contained and requires only Java standard library. Since v1.4, Smile may optionally leverage native BLAS/LAPACK library too. It also provides high-level operators in Scala and an interactive shell. In practice, data scientists usually build models with high-level tools such as R, Matlab, SAS, etc. However, developers have to spend a lot of time and energy to incorporate these models in the production system that are often implemented in general purpose programming languages such as Java and Scala. With Smile, data scientists and developers can work in the same environment to build machine learning applications quickly!

Download

Get Smile from the releases page of the project website. The universal tarball is also available and can be used on Mac, Linux and Windows.

If you would like to build Smile from source, please first install Java 17, Scala 2.13 and SBT 1.0+. Then clone the repo and build the package:

    $ git clone https://github.com/haifengl/smile.git
    $ cd smile
    $ sbt package
    

To build with Scala 3, run

    $ sbt ++3.3.3 scala/package
    

To test the latest code, run the following

    $ git pull
    $ bin/smile.sh
    

which will build the system and enter the Smile shell in Scala. If you prefer Java, you may run

    $ sbt shell/stage
    $ cd shell/target/universal/stage
    $ bin/jshell.sh
    

Shell

• Java
• Scala
• Kotlin

    $ bin/smile
    

    $ bin/smile -J-Xmx30G
    

                                                       ..::''''::..
                                                     .;''        ``;.
     ....                                           ::    ::  ::    ::
   ,;' .;:                ()  ..:                  ::     ::  ::     ::
   ::.      ..:,:;.,:;.    .   ::   .::::.         :: .:' ::  :: `:. ::
    '''::,   ::  ::  ::  `::   ::  ;:   .::        ::  :          :  ::
  ,:';  ::;  ::  ::  ::   ::   ::  ::,::''.         :: `:.      .:' ::
  `:,,,,;;' ,;; ,;;, ;;, ,;;, ,;;, `:,,,,:'          `;..``::::''..;'
                                                       ``::,,,,::''

  Welcome to Smile Shell; enter 'help<RETURN>' for list of supported commands.
  Type "exit<RETURN>" to leave the Smile Shell
  Version 4.0.0, Scala 2.13.14, SBT 1.9.9 built at 2024-07-03 09:52:24.404-0400
===============================================================================
smile>
    

    smile> smile.classification.r
    randomForest   rbfnet   rda
    

    smile> "Hello, World"
    res0: String = Hello, World

    smile> 2
    res1: Int = 2

    smile> 2+3
    res2: Int = 5
    

    smile> val x = 2 + 3
    x: Int = 5

    smile> print(x)
    5

    smile> val y = 2 * (x + 1)
    z: Int = 12
    

    smile> def sq(x: Double) = x * x
    sq: (x: Double)Double

    smile> sq(y)
    res4: Double = 441.0
    

    val data = read.arff(Paths.getTestData("weka/iris.arff"))
    println(data)

    val formula = "class" ~ "."
    val rf = smile.classification.randomForest(formula, data)
    println(s"OOB error = %.2f%%" format 100 * rf.error)
    

    $ bin/smile examples/iris.sc
    

    $ bin/jshell.sh
    

    $ bin/jshell.sh -R-Xmx30G
    

                                                       ..::''''::..
                                                     .;''        ``;.
     ....                                           ::    ::  ::    ::
   ,;' .;:                ()  ..:                  ::     ::  ::     ::
   ::.      ..:,:;.,:;.    .   ::   .::::.         :: .:' ::  :: `:. ::
    '''::,   ::  ::  ::  `::   ::  ;:   .::        ::  :          :  ::
  ,:';  ::;  ::  ::  ::   ::   ::  ::,::''.         :: `:.      .:' ::
  `:,,,,;;' ,;; ,;;, ;;, ,;;, ,;;, `:,,,,:'          `;..``::::''..;'
                                                       ``::,,,,::''
|  Welcome to Smile  -- Version  4.0.0
===============================================================================
|  Welcome to JShell -- Version 21.0.3
|  For an introduction type: /help intro

smile>
    

    smile> "Hello, World"
    $2 ==> "Hello, World"

    smile> 2
    $3 ==> 2

    smile> 2+3
    $4 ==> 5
    

    smile> var x = 2 + 3
    x ==> 5

    smile> var y = 2 * (x + 1)
    y ==> 12
    

    import smile.classification.RandomForest;
    import smile.data.formula.Formula;
    import smile.io.Read;
    import smile.util.Paths;

    var data = Read.arff(Paths.getTestData("weka/iris.arff"));
    System.out.println(data);

    var formula = Formula.lhs("class");
    var rf = RandomForest.fit(formula, data);
    System.out.println(rf.metrics());
    

    $ bin/jshell.sh examples/iris.jsh
    

    $ bin/kshell.sh
    

    $ bin/kshell.sh -J-Xmx30G
    

    Welcome to Kotlin version 2.0.0 (JRE 21.0.3+7-LTS-152)
    Type :help for help, :quit for quit
    >>>
    

    >>> "Hello, World"
    res0: kotlin.String = Hello, World
    >>> 2
    res1: kotlin.Int = 2
    >>> 2+3
    res2: kotlin.Int = 5
    

    >>> var x = 2 + 3
    >>> var y = 2 * (x + 1)
    >>> y
    res13: kotlin.Int = 12
    

    import smile.*
    import smile.classification.*
    import smile.data.formula.Formula
    import smile.util.Paths

    val data = read.arff(Paths.getTestData("weka/iris.arff"))
    println(data)

    val formula = Formula.lhs("class")
    val rf = randomForest(formula, data)
    println(rf.metrics())
    

    $ bin/kshell.sh -Xuse-fir-lt=false -script examples/iris.kts
    

Notebooks

You can also use Smile in your favorite Notebook. We recommend JupyterLab and provide jupyterlab.sh to setup the conda environment of Jupyter Lab for Smile with kernels for Scala and Kotlin. When you run jupyterlab.sh the first time, it will set up the environment automatically. You can update the environment with the option --update later when needed.

In Scala notebooks, it is helpful to add the following code to the notebook. We provide many notebook examples in the notebooks directory.

    import $ivy.`com.github.haifengl::smile-scala:3.1.1`

    import scala.language.postfixOps
    import org.apache.commons.csv.CSVFormat
    import smile._
    import smile.util._
    import smile.math._
    import smile.math.MathEx.{log2, logistic, factorial, lfactorial, choose, lchoose, random, randomInt, permutate, c, cbind, rbind, sum, mean, median, q1, q3, `var` => variance, sd, mad, min, max, whichMin, whichMax, unique, dot, distance, pdist, KullbackLeiblerDivergence => kld, JensenShannonDivergence => jsd, cov, cor, spearman, kendall, norm, norm1, norm2, normInf, standardize, normalize, scale, unitize, unitize1, unitize2, root}
    import smile.math.distance._
    import smile.math.kernel._
    import smile.math.matrix._
    import smile.math.matrix.Matrix._
    import smile.math.rbf._
    import smile.stat.distribution._
    import smile.data._
    import smile.data.formula._
    import smile.data.measure._
    import smile.data.`type`._
    import smile.json._
    import smile.interpolation._
    import smile.validation._
    import smile.association._
    import smile.base.cart.SplitRule
    import smile.base.mlp._
    import smile.base.rbf.RBF
    import smile.classification._
    import smile.regression.{ols, ridge, lasso, svr, gpr}
    import smile.feature._
    import smile.clustering._
    import smile.vq._
    import smile.manifold._
    import smile.mds._
    import smile.sequence._
    import smile.projection._
    import smile.nlp._
    import smile.wavelet._
    

To plot data with Swing based functions in Notebook, run the below code first.

    import smile.plot.swing._
    import smile.plot.show
    import smile.plot.Render._
    

To use Vega based plot functions in Notebook, run the below code instead.

    import smile.plot.vega._
    import smile.plot.show
    import smile.plot.Render._
    

A Gentle Example

This example shows how to use Smile for predictive modeling from Java and Scala code. First, let's load the data. Smile provides a couple of parsers for popular data formats, such as Parquet, Avro, Arrow, SAS7BDAT, Weka's ARFF files, LibSVM's file format, delimited text files, JSON, and binary sparse data. These classes are in the package smile.io. In the following example, we use the ARFF parser to load the weather dataset:

• Java
• Scala

    import smile.io.*;
    var weather = Read.arff("data/weka/weather.nominal.arff");
          

    import smile.io._
    val weather = read.arff("data/weka/weather.nominal.arff")
    

Most Smile data parsers return a DataFrame object, which is immutable and contain a fixed number of named columns. We can also parse plain delimited text files and the parser automatically infer the schema. In the below, we load the USPS zip code handwriting dataset in a white space delimitered text file.

• Java
• Scala

    import org.apache.commons.csv.CSVFormat;

    var format = CSVFormat.DEFAULT.withDelimiter(' ');
    var zipTrain = Read.csv("data/usps/zip.train", format);
    var zipTest = Read.csv("data/usps/zip.test", format);
          

    val zipTrain = read.csv("data/usps/zip.train", delimiter = " ", header = false)
    val zipTest = read.csv("data/usps/zip.test", delimiter = " ", header = false)
    

Because this data doesn't have a header line, the parser will assign V1, V2, ... as the column names. In particular, the first column (V1) is the class label.

Smile implements a variety of classification and regression algorithms. In what follows, we train a random forest model on the USPS data. Random forest is an ensemble classifier that consists of many decision trees and outputs the majority vote of individual trees. The method combines bagging idea and the random selection of features.

• Java
• Scala

    import smile.classification.*;
    import smile.data.formula.Formula;

    var formula = Formula.lhs("V1");
    
    var prop = new java.util.Properties();
    prop.setProperty("smile.random.forest.trees", "200");
    var forest = RandomForest.fit(formula, zipTrain, prop);
    System.out.println(forest.metrics());
          

    val formula: Formula = "V1" ~ "."

    val forest = randomForest(formula, zipTrain, ntrees = 200)
    println(forest.metrics())
    

In the example, we firstly define a Formula object, which specifies the model in a symbolic way. The left-hand-side (LHS) of formula is the response variable, and the right-hand-side (RHS) is a list of terms as independent variables. When the RHS is not specified, the rest of columns in the data frame are used by default. In the simpliest case, the terms (both of LHS and of RHS) are column names. But they can be functions (e.g. log) and transformations (e.g. interaction and factor crossing) too. The functions/transformations are symbolic and thus lazy.

With random forest, we may estimate the model accuracy with out-of-bag (OOB) samples. This is useful especially when we don't have a separate test dataset.

Now let's train a support vector machine (SVM) on the USPS data. As SVM is a kernel learning machine, it can be applied on any type of data as long as we can define a Mercer kernel on the data. Therefore, SVM class doesn't take a DataFrame as input but a generic array. We can leverage the formula object to extract the training samples and labels.

• Java
• Scala

    var x = formula.x(zipTrain).toArray();
    var y = formula.y(zipTrain).toIntArray();
    var testx = formula.x(zipTest).toArray();
    var testy = formula.y(zipTest).toIntArray();
          

    val x = formula.x(zipTrain).toArray()
    val y = formula.y(zipTrain).toIntArray()
    val testx = formula.x(zipTest).toArray()
    val testy = formula.y(zipTest).toIntArray()
    

The SVM employs a Gaussian kernel and one-to-one strategy as this is a multi-class problem. We also evaluate the model on the test data with Validation class, which provides a variety of model validation methods such as cross validation, bootstrap, etc.

• Java
• Scala

    import smile.math.kernel.GaussianKernel;
    import smile.validation.*;

    var kernel = new GaussianKernel(8.0);
    var svm = OneVersusOne.fit(x, y, (x, y) -> SVM.fit(x, y, kernel, 5, 1E-3));
    var pred = svm.predict(testx);
    System.out.format("Accuracy = %.2f%%%n", (100.0 * Accuracy.of(testy, pred)));
    System.out.format("Confusion Matrix: %s%n", ConfusionMatrix.of(testy, pred));
          

    val kernel = new GaussianKernel(8.0)
    val svm = ovo(x, y) { (x, y) =>
      SVM.fit(x, y, kernel, 5, 1E-3)
    }
    

Lastly, we will train a 5-layer deep learning model. Deep learning requires the features properly scaled/standardized. In this example, we employ the class Standardizer to transforms features to 0 mean and unit variance. An alternative is to subtract the median and divide by the IQR, which is implemented RobustStandardizer.

• Java
• Scala

    import smile.base.mlp.Layer;
    import smile.base.mlp.OutputFunction;
    import smile.classification.MLP;
    import smile.math.MathEx;
    
    var net = new MLP(Layer.input(256),
      Layer.sigmoid(768),
      Layer.sigmoid(192),
      Layer.sigmoid(30),
      Layer.mle(10, OutputFunction.SIGMOID)
    );

    net.setLearningRate(TimeFunction.linear(0.01, 20000, 0.001));
    
    for (int epoch = 0; epoch < 10; epoch++) {
      System.out.format("----- epoch %d -----%n", epoch);
      for (int i : MathEx.permutate(x.length)) {
        net.update(x[i], y[i]);
      }
      var prediction = net.predict(testx);
      System.out.format("Accuracy = %.2f%%%n", (100.0 * Accuracy.of(testy, prediction)));
    }
          

    val net = new MLP(Layer.input(256),
      Layer.sigmoid(768),
      Layer.sigmoid(192),
      Layer.sigmoid(30),
      Layer.mle(10, OutputFunction.SIGMOID)
    )

    net.setLearningRate(TimeFunction.linear(0.01, 20000, 0.001));
    
    (0 until 10).foreach(epoch => {
      println("----- epoch %d -----" format epoch)
      MathEx.permutate(x.length).foreach(i =>
        net.update(x(i), y(i))
      )
      val prediction = net.predict(testx)
      println("Accuracy = %.2f%%" format (100.0 * Accuracy.of(testy, prediction)))
    })
    

To use the trained model, we can apply the method predict on a new sample. Besides just returning class label, many methods (e.g. neural networks) can also output the posteriori probabilities of each class.

• Java
• Scala

    var posteriori = new double[10];
    forest.predict(zipTest.get(0), posteriori);
    svm.predict(testx[0]);
    net.predict(testx[0], posteriori);
          

    val posteriori = new Array[Double](10)
    forest.predict(zipTest.get(0), posteriori)
    svm.predict(testx(0))
    net.predict(testx(0), posteriori)