[graalvm-dev] Proposal for adding more labels to upstream Graal issues to ease filtering/parsing

[Foivos Zakkak]

Hello all,

In Mandrel we are facing some difficulties when trying to filter issues and
PRs from upstream Graal and we would like to start a discussion about
how to improve the current state.

In summary from the Mandrel perspective our end goal is to be able to
answer the following question:
Which issues reported upstream affect mandrel version X?

And we would like to be able to programmatically get those issues and
see which of them have been fixed and which are still open.

# Current state

Right now it appears like the whole workflow is focusing only on the
master branch. However, Mandrel being a downstream distribution also
needs to focus on older versions as well. In addition to Mandrel being
able to answer the above question (for GraalVM CE) would also be
valuable for filtering issues that my be affecting some GraalVM CE LTS 
release.

Currently the GraalVM team is actively using the following labels
(usually added by the Issue template) that indicate that an issue might be
Mandrel-releated:

+ bug
+ native-image

However, under these tags there is a large variety of issues some of 
them not
related to Mandrel. For instance Mandrel doesn't support JDK 8, so any 
issue
that is JDK 8 specific is of no interest to Mandrel.
(There is a label 'Java8', but it has been used only once.)

In addition to the aforementioned labels the issue template for
native-image bugs also asks for the following info:

     GraalVM version, or commit id if built from source: [e.g. EE 19.3, 
CE 20.0, CE 20.1.0-dev-20200304_0848]
     JDK major version: [e.g.:8]
     OS: [e.g. macOS Catalina]
     Architecture: [e.g.: AMD64]

In theory, if properly filled, the info provided in this form could 
cover our
needs. Unfortunately though some reporters may not take the time to
properly fill the form [1], or even if they do they might choose different
wording than the expected one (i.e., Intel instead of AMD64 [2]).

# Improving the current state

One way to work around the form-related issues, is to make the
graalvm-bot automatically ask reporters to properly fill the info if it
contains unexpected or missing entries.

Even with the form correctly filled though, reporters usually won't test the
bug against different setups.  As a result, an issue reported as Java 8
specific may be affecting Java 11 as well. So there is the need for
additional work to  properly identify interesting issues.

One approach would be to have a bot parse the aforementioned forms and
automatically add labels to the corresponding issue.
Then community members with "Triage" role [3] can go through issues
(possibly labeled with some label like
pending-triage) and fix/amend labels accordingly.

The main labels that appear to be missing and would allow us to create the
queries we want are:

- jdk-8, jdk-11  (plus jdk-9, jdk-10, jdk-11, jdk-12, etc. maybe)
   Here we could possibly stick to the existing Java8 and add Java11 etc.
- affects-19.3.1, affects-20.1.0, affects-20.2.0, etc.

Other than those the following would be "nice to have":

- platform-windows, platform-linux, platform-darwin (they already exist 
but are not being used)
- amd64, aarch64, etc.

What are your thoughts?

Thanks for reading that far :),
Foivos

[1] https://github.com/oracle/graal/issues/2902
[2] https://github.com/oracle/graal/issues/2877
[3]
https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-organizations-and-teams/repository-permission-levels-for-an-organization#permission-levels-for-repositories-owned-by-an-organization

-- 
Foivos Zakkak
Senior Software Engineer, R&D Middleware
Red Hat
7B40 69D9 29BA AE91 C0B3 220A 0846 BFD1 03F0 4EA1


-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xCC102CF429A69A3B.asc
Type: application/pgp-keys
Size: 1758 bytes
Desc: not available
Url : http://oss.oracle.com/pipermail/graalvm-dev/attachments/20201110/a61137f3/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xCC102CF429A69A3B.asc
Type: application/pgp-keys
Size: 1757 bytes
Desc: not available
Url : http://oss.oracle.com/pipermail/graalvm-dev/attachments/20201110/a61137f3/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
Url : http://oss.oracle.com/pipermail/graalvm-dev/attachments/20201110/a61137f3/attachment-0002.bin