Quick start guide

Quick Example

Let's build a classifier for the classic iris dataset. If you don't have RDatasets, Pkg.add it.

julia> using RDatasets: dataset

julia> iris = dataset("datasets", "iris");

julia> first(iris, 5)
5×5 DataFrames.DataFrame
│ Row │ SepalLength │ SepalWidth │ PetalLength │ PetalWidth │ Species │
│     │ Float64     │ Float64    │ Float64     │ Float64    │ Cat…    │
├─────┼─────────────┼────────────┼─────────────┼────────────┼─────────┤
│ 1   │ 5.1         │ 3.5        │ 1.4         │ 0.2        │ setosa  │
│ 2   │ 4.9         │ 3.0        │ 1.4         │ 0.2        │ setosa  │
│ 3   │ 4.7         │ 3.2        │ 1.3         │ 0.2        │ setosa  │
│ 4   │ 4.6         │ 3.1        │ 1.5         │ 0.2        │ setosa  │
│ 5   │ 5.0         │ 3.6        │ 1.4         │ 0.2        │ setosa  │

ScikitLearn.jl expects arrays, but DataFrames can also be used - see the corresponding section of the manual

julia> X = convert(Array, iris[!, [:SepalLength, :SepalWidth, :PetalLength, :PetalWidth]]) #subset of original data not a copy
150×4 Array{Float64,2}:
 5.1  3.5  1.4  0.2
 4.9  3.0  1.4  0.2
 4.7  3.2  1.3  0.2
 4.6  3.1  1.5  0.2
 5.0  3.6  1.4  0.2
 5.4  3.9  1.7  0.4
 4.6  3.4  1.4  0.3
 5.0  3.4  1.5  0.2
 4.4  2.9  1.4  0.2
 4.9  3.1  1.5  0.1
 ⋮
 6.9  3.1  5.1  2.3
 5.8  2.7  5.1  1.9
 6.8  3.2  5.9  2.3
 6.7  3.3  5.7  2.5
 6.7  3.0  5.2  2.3
 6.3  2.5  5.0  1.9
 6.5  3.0  5.2  2.0
 6.2  3.4  5.4  2.3
 5.9  3.0  5.1  1.8

julia> y = convert(Array, iris[!, :Species])
150-element Array{String,1}:
 "setosa"
 "setosa"
 "setosa"
 "setosa"
 "setosa"
 "setosa"
 "setosa"
 "setosa"
 "setosa"
 "setosa"
 ⋮
 "virginica"
 "virginica"
 "virginica"
 "virginica"
 "virginica"
 "virginica"
 "virginica"
 "virginica"
 "virginica"

Next, we load the LogisticRegression model from scikit-learn's library. This would require scikit-learn. See Installation

julia> using ScikitLearn

julia> @sk_import linear_model: LogisticRegression
PyObject <class 'sklearn.linear_model._logistic.LogisticRegression'>

Every model's constructor accepts hyperparameters (such as regression strength, whether to fit the intercept, the penalty type, etc.) as keyword arguments. Check out ?LogisticRegression for details.

julia> model = LogisticRegression(fit_intercept=true, max_iter = 200)
PyObject LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
                   intercept_scaling=1, l1_ratio=None, max_iter=200,
                   multi_class='auto', n_jobs=None, penalty='l2',
                   random_state=None, solver='lbfgs', tol=0.0001, verbose=0,
                   warm_start=False)

Then we train the model and evaluate its accuracy on the training set:

julia> fit!(model, X, y);

julia> accuracy = score(model, X, y)
0.9733333333333334

julia> println("accuracy: $accuracy")
accuracy: 0.9733333333333334

Cross-validation

This will train five models, on five train/test splits of X and y, and return the test-set accuracy of each:

julia> using ScikitLearn.CrossValidation: cross_val_score

julia> cross_val_score(LogisticRegression(max_iter=130), X, y; cv=5)  # 5-fold
5-element Array{Float64,1}:
 0.9666666666666667
 1.0
 0.9333333333333333
 0.9666666666666667
 1.0
  

See this tutorial for more information.

Hyperparameter tuning

LogisticRegression has a regularization-strength parameter C (smaller is stronger). We can use grid search algorithms to find the optimal C.

GridSearchCV will try all values of C in 0.1:0.1:2.0 and will return the one with the highest cross-validation performance.

julia> using ScikitLearn.GridSearch: GridSearchCV

julia> gridsearch = GridSearchCV(LogisticRegression(max_iter=200), Dict(:C => 0.1:0.1:2.0))
GridSearchCV
  estimator: PyCall.PyObject
  param_grid: Dict{Symbol,StepRangeLen{Float64,Base.TwicePrecision{Float64},Base.TwicePrecision{Float64}}}
  scoring: Nothing nothing
  loss_func: Nothing nothing
  score_func: Nothing nothing
  fit_params: Dict{Any,Any}
  n_jobs: Int64 1
  iid: Bool true
  refit: Bool true
  cv: Nothing nothing
  verbose: Int64 0
  error_score: String "raise"
  scorer_: Nothing nothing
  best_params_: Nothing nothing
  best_score_: Nothing nothing
  grid_scores_: Nothing nothing
  best_estimator_: Nothing nothing

julia> fit!(gridsearch, X, y);

julia> println("Best parameters: $(gridsearch.best_params_)")
Best parameters: Dict{Symbol,Any}(:C => 0.6)

Finally, we plot cross-validation accuracy vs. C

julia> using PyPlot, Statistics

julia> plot([cv_res.parameters[:C] for cv_res in gridsearch.grid_scores_],
            [mean(cv_res.cv_validation_scores) for cv_res in gridsearch.grid_scores_])
1-element Array{PyCall.PyObject,1}:
 PyObject <matplotlib.lines.Line2D object at 0x7f7e8aef6c10>

Saving the model to disk

Both Python and Julia models can be saved to disk

julia> import JLD, PyCallJLD

julia> JLD.save("my_model.jld", "model", model)

julia> model = JLD.load("my_model.jld", "model")    # Load it back
┌ Warning: type PyCallJLD.PyObjectSerialization not present in workspace; reconstructing
└ @ JLD ~/.julia/packages/JLD/uVJmd/src/jld_types.jl:722
getfield(JLD, Symbol("##PyCallJLD.PyObjectSerialization#371"))(UInt8[0x80, 0x03, 0x63, 0x73, 0x6b, 0x6c, 0x65, 0x61, 0x72, 0x6e  …  0x2e, 0x32, 0x32, 0x2e, 0x31, 0x71, 0x43, 0x75, 0x62, 0x2e])