[tribuo-devel] [EXT]:Re: [External] : Re: Tribuo Non Numerical Features Support

[Adam Pocock]

Hi Sandeep,

Tribuo doesn’t support inference for LightGBM models, aside from those which have been exported in ONNX format. You’d have to look at how the ONNX format model expects the categorical inputs to be passed in, I’m not sure if ONNX’s TreeEnsembleRegressor (https://github.com/onnx/onnx/blob/main/docs/Operators-ml.md#aionnxmltreeensembleregressor) supports special treatment of categorical inputs so it might not be possible. It looks like there is support for LightGBM in things like JPMML, but JPMML is AGPL licensed and so we can’t use it in Tribuo. We haven’t looked at directly wrapping the LightGBM Java API, as we already had XGBoost support for tree ensembles.

Thanks,

Adam
--
Adam Pocock
Principal Member of Technical Staff
Machine Learning Research Group
Oracle Labs, Burlington, MA

On 16 Jun 2023, at 11:08, Sikka, Sandeep <sandeep.sikka at staples.com> wrote:

Hi Adam
Thank you for the quick response. We are looking to score (perform inference) on a LGBMRegressor<https://urldefense.com/v3/__https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMRegressor.html__;!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7Jy9qPBBw$> trained externally with the categorical features applied using the categorical_features parameter: https://lightgbm.readthedocs.io/en/latest/Advanced-Topics.html#categorical-feature-support<https://urldefense.com/v3/__https://lightgbm.readthedocs.io/en/latest/Advanced-Topics.html*categorical-feature-support__;Iw!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7LbplBZGQ$>. This uses a technique different from OHE: https://lightgbm.readthedocs.io/en/latest/Features.html#optimal-split-for-categorical-features<https://urldefense.com/v3/__https://lightgbm.readthedocs.io/en/latest/Features.html*optimal-split-for-categorical-features__;Iw!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7L6i0QwjQ$>. Do you have any thoughts on how we can achieve inference in Tribuo (we train the model in Python scikit-learn pipelines)?

Thanks
Sandeep.
From: Adam Pocock <adam.pocock at oracle.com<mailto:adam.pocock at oracle.com>>
Date: Friday, June 16, 2023 at 9:34 AM
To: "Sikka, Sandeep" <sandeep.sikka at staples.com<mailto:sandeep.sikka at staples.com>>, "Parthasarathy, Bharadwaj" <bharadwaj.parthasarathy at staples.com<mailto:bharadwaj.parthasarathy at staples.com>>
Cc: "tribuo-devel at oss.oracle.com<mailto:tribuo-devel at oss.oracle.com>" <tribuo-devel at oss.oracle.com<mailto:tribuo-devel at oss.oracle.com>>, "Kumar, Navdeep" <navdeep.kumar at staples.com<mailto:navdeep.kumar at staples.com>>, Jack Sullivan <jack.t.sullivan at oracle.com<mailto:jack.t.sullivan at oracle.com>>
Subject: [EXT]:Re: [External] : Re: Tribuo Non Numerical Features Support

Hi Sandeep, Bharad,

The Tribuo mailing list is monitored, but I can’t see the email you sent to the list in the moderator console nor the archives, did you get a bounceback from the mailserver? It may be configured only to accept emails from people who have joined the mailing list. Anyway, the Tribuo development team have their email addresses listed in the pom file visible on Github.

As to the question of feature support, in Tribuo all features are converted to numerical values during the data loading step. Categoricals can be encoded as one hot features or directly as doubles by choosing an index, and ordinals can be encoded as doubles. Very few ML algorithms directly support categorical variables without one hot-encoding or vectorization, I think of the popular ones the only kind that do are trees. Tribuo’s tree package doesn’t currently have specializations for categoricals though there is enough information in the feature domain to support that if we do add the algorithmic support. In tree algorithms ordinals are naturally supported as the splits don’t care about the distance between any feature values, only their relative ordering (or at least all the splitting algorithms we have in Tribuo are of that kind). You can see how to perform featurization into doubles in a number of ways in the Columnar Data (https://tribuo.org/learn/4.3/tutorials/columnar-tribuo-v4.html<https://urldefense.com/v3/__https://tribuo.org/learn/4.3/tutorials/columnar-tribuo-v4.html__;!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7J1rg3Sxg$>) and Document Classification (https://tribuo.org/learn/4.3/tutorials/document-classification-tribuo-v4.html<https://urldefense.com/v3/__https://tribuo.org/learn/4.3/tutorials/document-classification-tribuo-v4.html__;!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7LtOLJzVw$>) tutorials.

Thanks,

Adam
--
Adam Pocock
Principal Member of Technical Staff
Machine Learning Research Group
Oracle Labs, Burlington, MA


On 16 Jun 2023, at 00:13, Sikka, Sandeep <sandeep.sikka at staples.com<mailto:sandeep.sikka at staples.com>> wrote:

Adding a few individual emails on the project from Github. The devel email list doesn’t appear to have any activity.

Thanks
Sandeep Sikka
Distinguished Engineer
500 Staples Dr., Framingham MA
508.253.7977
sandeep.sikka at staples.com<mailto:sandeep.sikka at staples.com>

<image001.png>

<image002.png><image003.png> <image004.png> <image005.png>



From: "Parthasarathy, Bharadwaj" <bharadwaj.parthasarathy at staples.com<mailto:bharadwaj.parthasarathy at staples.com>>
Date: Thursday, June 15, 2023 at 2:46 PM
To: "tribuo-devel at oss.oracle.com<mailto:tribuo-devel at oss.oracle.com>" <tribuo-devel at oss.oracle.com<mailto:tribuo-devel at oss.oracle.com>>
Cc: "Sikka, Sandeep" <Sandeep.Sikka at Staples.com<mailto:Sandeep.Sikka at Staples.com>>, "Kumar, Navdeep" <navdeep.kumar at staples.com<mailto:navdeep.kumar at staples.com>>
Subject: Tribuo Non Numerical Features Support

Hi,


This is Bharadwaj from one of Staples' Engineering teams. We are exploring Tribuo java framework for one of our use cases. We are trying to use Tribuo for runtime inferencing with a regressor model.

We have non numerical features that we are trying to integrate with Tribuo. However, all the classes or samples for regressor points us to passing feature values as only double. Feature names as String and Feature Values as Double is what we see.

Does Tribuo support non numerical features (categorical and ordinal features) such as String, Boolean etc or is it strictly only double values? How can we train and infer supervised models that consume categorical and ordinal features?

I am not sure if we are missing something here, it would be helpful if you can provide your inputs.


Regards,
Bharad


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.oracle.com/pipermail/tribuo-devel/attachments/20230616/2b2e3eab/attachment-0001.html>