<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
Hi Sandeep,
<div><br>
</div>
<div>Tribuo doesn’t support inference for LightGBM models, aside from those which have been exported in ONNX format. You’d have to look at how the ONNX format model expects the categorical inputs to be passed in, I’m not sure if ONNX’s TreeEnsembleRegressor
(<a href="https://github.com/onnx/onnx/blob/main/docs/Operators-ml.md#aionnxmltreeensembleregressor">https://github.com/onnx/onnx/blob/main/docs/Operators-ml.md#aionnxmltreeensembleregressor</a>) supports special treatment of categorical inputs so it might
not be possible. It looks like there is support for LightGBM in things like JPMML, but JPMML is AGPL licensed and so we can’t use it in Tribuo. We haven’t looked at directly wrapping the LightGBM Java API, as we already had XGBoost support for tree ensembles.</div>
<div><br>
</div>
<div>Thanks,</div>
<div><br>
</div>
<div>Adam<br>
<div>
<div>--<br>
Adam Pocock<br>
Principal Member of Technical Staff<br>
Machine Learning Research Group<br>
Oracle Labs, Burlington, MA</div>
</div>
<div><br>
<blockquote type="cite">
<div>On 16 Jun 2023, at 11:08, Sikka, Sandeep <sandeep.sikka@staples.com> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Hi Adam<o:p></o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Thank you for the quick response. We are looking to score (perform inference) on a<span class="Apple-converted-space"> </span><a href="https://urldefense.com/v3/__https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMRegressor.html__;!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7Jy9qPBBw$" style="color: blue; text-decoration: underline;">LGBMRegressor</a><span class="Apple-converted-space"> </span>trained
externally with the categorical features applied using the categorical_features parameter:<span class="Apple-converted-space"> </span><a href="https://urldefense.com/v3/__https://lightgbm.readthedocs.io/en/latest/Advanced-Topics.html*categorical-feature-support__;Iw!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7LbplBZGQ$" style="color: blue; text-decoration: underline;">https://lightgbm.readthedocs.io/en/latest/Advanced-Topics.html#categorical-feature-support</a>.
This uses a technique different from OHE:<span class="Apple-converted-space"> </span><a href="https://urldefense.com/v3/__https://lightgbm.readthedocs.io/en/latest/Features.html*optimal-split-for-categorical-features__;Iw!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7L6i0QwjQ$" style="color: blue; text-decoration: underline;">https://lightgbm.readthedocs.io/en/latest/Features.html#optimal-split-for-categorical-features</a>.
Do you have any thoughts on how we can achieve inference in Tribuo (we train the model in Python scikit-learn pipelines)?<o:p></o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Thanks<o:p></o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Sandeep.<span class="Apple-converted-space"> </span><o:p></o:p></div>
<div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentcolor currentcolor; border-image: none; padding: 3pt 0in 0in;">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 12pt;">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 12pt;">Adam Pocock <<a href="mailto:adam.pocock@oracle.com" style="color: blue; text-decoration: underline;">adam.pocock@oracle.com</a>><br>
<b>Date:<span class="Apple-converted-space"> </span></b>Friday, June 16, 2023 at 9:34 AM<br>
<b>To:<span class="Apple-converted-space"> </span></b>"Sikka, Sandeep" <<a href="mailto:sandeep.sikka@staples.com" style="color: blue; text-decoration: underline;">sandeep.sikka@staples.com</a>>, "Parthasarathy, Bharadwaj" <<a href="mailto:bharadwaj.parthasarathy@staples.com" style="color: blue; text-decoration: underline;">bharadwaj.parthasarathy@staples.com</a>><br>
<b>Cc:<span class="Apple-converted-space"> </span></b>"<a href="mailto:tribuo-devel@oss.oracle.com" style="color: blue; text-decoration: underline;">tribuo-devel@oss.oracle.com</a>" <<a href="mailto:tribuo-devel@oss.oracle.com" style="color: blue; text-decoration: underline;">tribuo-devel@oss.oracle.com</a>>,
"Kumar, Navdeep" <<a href="mailto:navdeep.kumar@staples.com" style="color: blue; text-decoration: underline;">navdeep.kumar@staples.com</a>>, Jack Sullivan <<a href="mailto:jack.t.sullivan@oracle.com" style="color: blue; text-decoration: underline;">jack.t.sullivan@oracle.com</a>><br>
<b>Subject:<span class="Apple-converted-space"> </span></b>[EXT]:Re: [External] : Re: Tribuo Non Numerical Features Support<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div>
</div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Hi Sandeep, Bharad,<span class="Apple-converted-space"> </span><o:p></o:p></div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">The Tribuo mailing list is monitored, but I can’t see the email you sent to the list in the moderator console nor the archives, did you get a bounceback from the mailserver? It may
be configured only to accept emails from people who have joined the mailing list. Anyway, the Tribuo development team have their email addresses listed in the pom file visible on Github.<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">As to the question of feature support, in Tribuo all features are converted to numerical values during the data loading step. Categoricals can be encoded as one hot features or directly
as doubles by choosing an index, and ordinals can be encoded as doubles. Very few ML algorithms directly support categorical variables without one hot-encoding or vectorization, I think of the popular ones the only kind that do are trees. Tribuo’s tree package
doesn’t currently have specializations for categoricals though there is enough information in the feature domain to support that if we do add the algorithmic support. In tree algorithms ordinals are naturally supported as the splits don’t care about the distance
between any feature values, only their relative ordering (or at least all the splitting algorithms we have in Tribuo are of that kind). You can see how to perform featurization into doubles in a number of ways in the Columnar Data (<a href="https://urldefense.com/v3/__https://tribuo.org/learn/4.3/tutorials/columnar-tribuo-v4.html__;!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7J1rg3Sxg$" style="color: blue; text-decoration: underline;">https://tribuo.org/learn/4.3/tutorials/columnar-tribuo-v4.html</a>)
and Document Classification (<a href="https://urldefense.com/v3/__https://tribuo.org/learn/4.3/tutorials/document-classification-tribuo-v4.html__;!!ACWV5N9M2RV99hQ!MaIMJSbVSuFi4H4SnXBIfhxdZCWgg2GBNhEefwQDhe4vef4HJeHRLEbwfLxphLQ8iFNWOqXkTwp7VTblc7LtOLJzVw$" style="color: blue; text-decoration: underline;">https://tribuo.org/learn/4.3/tutorials/document-classification-tribuo-v4.html</a>)
tutorials.<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Thanks,<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Adam<o:p></o:p></div>
</div>
<div>
<div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">--<br>
Adam Pocock<br>
Principal Member of Technical Staff<br>
Machine Learning Research Group<br>
Oracle Labs, Burlington, MA<o:p></o:p></div>
</div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><br>
<br>
<o:p></o:p></div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" type="cite">
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">On 16 Jun 2023, at 00:13, Sikka, Sandeep <<a href="mailto:sandeep.sikka@staples.com" style="color: blue; text-decoration: underline;">sandeep.sikka@staples.com</a>> wrote:<o:p></o:p></div>
</div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div>
<div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Adding a few individual emails on the project from Github. The devel email list doesn’t appear to have any activity.<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Thanks<o:p></o:p></div>
</div>
<div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 14pt; font-family: Arial, sans-serif; color: rgb(203, 0, 10);">Sandeep Sikka</span></b><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 10.5pt; font-family: Arial, sans-serif; color: rgb(100, 100, 100);">Distinguished Engineer</span></b><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10.5pt; font-family: Arial, sans-serif; color: rgb(100, 100, 100);">500 Staples Dr., Framingham MA</span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10.5pt; font-family: Arial, sans-serif; color: rgb(100, 100, 100);">508.253.7977</span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 10.5pt; font-family: Arial, sans-serif;"><a href="mailto:sandeep.sikka@staples.com" style="color: blue; text-decoration: underline;">sandeep.sikka@staples.com</a></span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; line-height: 18pt;">
<span style="font-size: 3pt; font-family: Arial, sans-serif; color: rgb(100, 100, 100);"> </span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><sub><span style="font-family: Arial, sans-serif; color: rgb(100, 100, 100);"><span id="cid:image001.png@01D9A042.EB977E40"><image001.png></span></span></sub><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 3pt; font-family: Arial, sans-serif; color: rgb(100, 100, 100);"> </span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><a name="_Hlk3999959"><span style="font-family: Arial, sans-serif; color: rgb(100, 100, 100);"><span id="cid:image002.png@01D9A042.EB977E40"><image002.png></span><span id="cid:image003.png@01D9A042.EB977E40"><image003.png></span> <span id="cid:image004.png@01D9A042.EB977E40"><image004.png></span> <span id="cid:image005.png@01D9A042.EB977E40"><image005.png></span></span></a><span></span><o:p></o:p></div>
</div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div>
</div>
<div style="border-width: 1pt medium medium; border-style: solid none none; padding: 3pt 0in 0in; border-color: currentcolor; border-image: none;">
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 12pt;">From:<span class="apple-converted-space"> </span></span></b><span style="font-size: 12pt;">"Parthasarathy, Bharadwaj" <<a href="mailto:bharadwaj.parthasarathy@staples.com" style="color: blue; text-decoration: underline;">bharadwaj.parthasarathy@staples.com</a>><br>
<b>Date:<span class="apple-converted-space"> </span></b>Thursday, June 15, 2023 at 2:46 PM<br>
<b>To:<span class="apple-converted-space"> </span></b>"<a href="mailto:tribuo-devel@oss.oracle.com" style="color: blue; text-decoration: underline;">tribuo-devel@oss.oracle.com</a>" <<a href="mailto:tribuo-devel@oss.oracle.com" style="color: blue; text-decoration: underline;">tribuo-devel@oss.oracle.com</a>><br>
<b>Cc:<span class="apple-converted-space"> </span></b>"Sikka, Sandeep" <<a href="mailto:Sandeep.Sikka@Staples.com" style="color: blue; text-decoration: underline;">Sandeep.Sikka@Staples.com</a>>, "Kumar, Navdeep" <<a href="mailto:navdeep.kumar@staples.com" style="color: blue; text-decoration: underline;">navdeep.kumar@staples.com</a>><br>
<b>Subject:<span class="apple-converted-space"> </span></b>Tribuo Non Numerical Features Support</span><o:p></o:p></div>
</div>
</div>
<div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div>
</div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;">Hi,<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;"> <o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;"> <o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;">This is Bharadwaj from one of Staples' Engineering teams. We are exploring Tribuo java framework for one of our use cases. We
are trying to use Tribuo for runtime inferencing with a regressor model.<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;"> <o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;">We have non numerical features that we are trying to integrate with Tribuo. However, all the classes or samples for regressor
points us to passing feature values as only double. Feature names as String and Feature Values as Double is what we see.<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;"> <o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;">Does Tribuo support non numerical features<span class="apple-converted-space"> </span><u>(categorical and ordinal features)</u><span class="apple-converted-space"> </span>such
as String, Boolean etc or is it strictly only double values? How can we train and infer supervised models that consume categorical and ordinal features?<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;"> <o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;">I am not sure if we are missing something here, it would be helpful if you can provide your inputs.<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"> <o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Regards,<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">Bharad<o:p></o:p></div>
</div>
</div>
</blockquote>
</div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</body>
</html>