[graalvm-users] help for native images
Oleg Šelajev
OLEG.SELAJEV at ORACLE.COM
Tue Jan 28 05:26:09 PST 2020
Hi René!
Sorry for a long response time, I read your email, started answering and then got distracted. Sorry!
Yeah, you can raise an issue, perhaps that would be more appropriate, since you have code that you want to make working.
However, let me still quickly answer the initial questions from your first email:
>> 1) what is actually executed from a native-image generated executable fallback image. Not my classes?
Your classes, technically fallback image is not something you want. It links in HotSpot so it doesn't give you any benefits of the native image. Might as well ship HotSpot with your code and be done. That's why you see problems with moving this executable, because it remembers the locations of the files and whatnot. In short, I don't want to assume your use case, but I don't think you want the fallback image. That's why when you change the class files the changes are reflected, because the linked hotspot loads the classes and executes them normally.
>> 2) how to go about generating a fully self-contained image - that error message does not exactly pinpoint (at all) what the problem is doing this;
If you want the self-contained native image (maybe not statically linked, but say against libc like a normal native app) -- you need to run the native image with --no-fallback and resolve everything that prevents it from finishing successfully.
>> 3) can I somehow list the symbols in the executable - I did an objdump but I see generic C-library functions and none of my own classes - but a System.getProperty(“java.version”) tells me 8 - while running on a Java 13 JVM.
Symbols -- don't know from the top of my head, like with any other binary file I presume. If you built the native image with JDK8 based GraalVM, it linked HotSpot from that distribution. Not sure what do you mean running on Java 13 here.
>> 4) how can the runtime know it is a native executable and no class running in a conventional JVM - this would probably be enough for me to differentiate
I don't understand the question, but I hope the answers above clear it a bit. It does run with HotSpot.
>> 5) I see one can make shared object libraries. That would be great, because I have a few entrypoints (nrc, pipe, pipc, nrws) that all uses the same classes. But how to use this? Is there an example?
You need to annotate your Java code with the @CEntryPoint(name = "name") annotation then the native image utility will expose it as the entry point and you can call it. For example here, part 8. Java code as a native library https://medium.com/graalvm/graalvm-ten-things-12d9111f307d <https://medium.com/graalvm/graalvm-ten-things-12d9111f307d>
>> 6) (bonus points) I miss an aarch64 native-image version of JVM8 - is there any chance of this? I am fine with 64-bit only and Raspbian should catch up quickly, but NetRexx on Ubuntu on the Raspi 4 with Java 11 would be great - need a Graal native-image 8 for that. Or do I?
You'd need the jdk8 based GraalVM for that, afaik our distributions currently for aarch are java 11. But you should be able technically create a native image of an app with JDK11 based GraalVM for aarch and run it on raspberry pi.
Regarding the second email, and the issues with being stuck, sometimes the error messages from the native image utility are not the most straightforward to decipher. You should try the --no-fallback, I'd recommend you try generating the config for the resources with the javaagent as shown here: https://github.com/oracle/graal/blob/master/substratevm/CONFIGURE.md <https://github.com/oracle/graal/blob/master/substratevm/CONFIGURE.md>
I'd recommend first starting with translating everything to the bytecode and then feeding that to the native image utility, sounds like a simpler exercise.
Oleg
> On 27 Jan 2020, at 21:31, René Jansen <rvjansen at xs4all.nl> wrote:
>
> sadly replying to my own post now; I am a bit further.
>
> I discovered --report-unsupported-elements-at-runtime.
>
> Now I do a
>
> native-image -cp $(CLASSPATH) --report-unsupported-elements-at-runtime org.netrexx.process.NetRexxC
>
> Now an image of 10MB is built instead of the previous 3.8MB. It runs without the .jar now, so my conclusion is that the fallback image only instatiates a JVM and then branches to your main().
> The image does not work, and instead of reporting at runtime, it does nothing - when I hand it a source file to compile.
>
> I also discovered -H:+ReportExceptionStackTraces. It tells me about lambda’s when I don’t really do lambda’s.
>
> ➜ netrexx-code git:(master) ✗ make natives
> native-image -cp .:/Users/rvjansen/apps/netrexx-code/ant/jansi-1.17.jar:/Users/rvjansen/apps/netrexx-code/build/lib/NetRexxC.jar:/Users/rvjansen/lib/Enzo-0.3.6.jar:/Users/rvjansen/lib/uuid-3.2.jar:/Users/rvjansen/lib/commons-net-3.3.jar:/Users/rvjansen/papiamento/src/build:/Users/rvjansen/lib/swing-layout-1.0.4.jar:/Users/rvjansen/homebrew/Cellar/kotlin/1.2.10/libexec/lib/kotlin-runtime.jar:/Users/rvjansen/lib/junit-4.12.jar:/Users/rvjansen/lib/hamcrest-core-1.3.jar:/Users/rvjansen/lib/jogl.jar:/Users/rvjansen/lib/antlr-4.5.3-complete.jar:/Users/rvjansen/lib/sqlite-jdbc-3.21.0.1.jar:/Users/rvjansen/lib/postgresql-9.4.1211.jre7.jar:/Users/rvjansen/lib/mail.jar:/Users/rvjansen/lib/activation.jar:/Users/rvjansen/lib/log4j-1.2.16.jar:/Users/rvjansen/lib/fit.jar:/Users/rvjansen/lib/bpjtk-v3.0.2_b20081203.jar:/Users/rvjansen/lib/org.eclipse.paho.client.mqttv3-1.2.2.jar:/Users/rvjansen/lib/commons-codec-1.7.jar:/Users/rvjansen/lib/jsch-0.1.50.jar:/Users/rvjansen/lib/ibmjzos.jar:/Users/rvjansen/lib/jsp-api.jar:/Users/rvjansen/lib/servlet-api-3.0.jar:/Users/rvjansen/lib/java2nrx.jar:/Applications/SWI-Prolog.app/Contents/swipl/lib/jpl.jar:/Users/rvjansen/apps/graalvm-ce-java8-19.3.1/Contents/Home/jre/lib/ext/jfxrt.jar:/Users/rvjansen/lib/pivot-charts-2.0.5.jar:/Users/rvjansen/lib/pivot-core-2.0.5.jar:/Users/rvjansen/lib/pivot-web-2.0.5.jar:/Users/rvjansen/lib/pivot-web-server-2.0.5.jar:/Users/rvjansen/lib/pivot-wtk-2.0.5.jar:/Users/rvjansen/lib/pivot-wtk-terra-2.0.5.jar org.netrexx.process.NetRexxC -H:+ReportExceptionStackTraces
> Build on Server(pid: 82860, port: 55394)
> [org.netrexx.process.netrexxc:82860] classlist: 3,750.00 ms
> [org.netrexx.process.netrexxc:82860] (cap): 1,337.98 ms
> [org.netrexx.process.netrexxc:82860] setup: 1,573.24 ms
> [org.netrexx.process.netrexxc:82860] analysis: 32,304.55 ms
> Warning: Aborting stand-alone image build due to unsupported features
> com.oracle.svm.hosted.FallbackFeature$FallbackImageRequest: Aborting stand-alone image build due to unsupported features
> at com.oracle.svm.hosted.FallbackFeature.reportFallback(FallbackFeature.java:213)
> at com.oracle.svm.hosted.FallbackFeature.reportFallback(FallbackFeature.java:198)
> at com.oracle.svm.hosted.FallbackFeature.afterAnalysis(FallbackFeature.java:278)
> at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$9(NativeImageGenerator.java:724)
> at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:63)
> at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:724)
> at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:530)
> at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:445)
> at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
> at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
> at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
> Build on Server(pid: 82860, port: 55394)
> [org.netrexx.process.netrexxc:82860] classlist: 126.17 ms
> [org.netrexx.process.netrexxc:82860] (cap): 1,404.38 ms
> [org.netrexx.process.netrexxc:82860] setup: 1,604.35 ms
> [org.netrexx.process.netrexxc:82860] (typeflow): 2,153.07 ms
> [org.netrexx.process.netrexxc:82860] (objects): 2,408.80 ms
> [org.netrexx.process.netrexxc:82860] (features): 91.76 ms
> [org.netrexx.process.netrexxc:82860] analysis: 4,720.93 ms
> [org.netrexx.process.netrexxc:82860] (clinit): 101.57 ms
> [org.netrexx.process.netrexxc:82860] universe: 244.72 ms
> [org.netrexx.process.netrexxc:82860] (parse): 211.83 ms
> [org.netrexx.process.netrexxc:82860] (inline): 643.13 ms
> [org.netrexx.process.netrexxc:82860] (compile): 1,126.10 ms
> [org.netrexx.process.netrexxc:82860] compile: 2,148.40 ms
> [org.netrexx.process.netrexxc:82860] image: 241.02 ms
> [org.netrexx.process.netrexxc:82860] write: 143.82 ms
> [org.netrexx.process.netrexxc:82860] [total]: 9,291.60 ms
> Warning: Image 'org.netrexx.process.netrexxc' is a fallback image that requires a JDK for execution (use --no-fallback to suppress fallback image generation).
> mv org.netrexx.process.NetRexxC nrc
>
> I need to trace why it does not pickup the source file, while it does pick up an option like -help. I have to say it is very fast; but for now to no avail.
>
> In a possible related issue, I tried to native-compile hello world, it fails in the same manner.
>
> say ‘hello world’
>
> when compiling the hellonrx.class file, I get this:
>
> ➜ test git:(master) ✗ java org.netrexx.process.NetRexxC -replace -keepasjava -format hellonrx.nrx
> NetRexx portable processor 3.09-PRE build 137-20200127-1912
> Copyright (c) RexxLA, 2011,2019. All rights reserved.
> Parts Copyright (c) IBM Corporation, 1995,2008.
> Program hellonrx.nrx
> Compilation of 'hellonrx.nrx' successful
> ➜ test git:(master) ✗ native-image hellonrx -H:+ReportExceptionStackTraces
> Build on Server(pid: 83895, port: 55676)
> [hellonrx:83895] classlist: 130.94 ms
> [hellonrx:83895] (cap): 1,427.32 ms
> [hellonrx:83895] setup: 1,625.63 ms
> [hellonrx:83895] analysis: 5,080.18 ms
> Warning: Aborting stand-alone image build due to unsupported features
> com.oracle.svm.hosted.FallbackFeature$FallbackImageRequest: Aborting stand-alone image build due to unsupported features
> at com.oracle.svm.hosted.FallbackFeature.reportFallback(FallbackFeature.java:213)
> at com.oracle.svm.hosted.FallbackFeature.reportFallback(FallbackFeature.java:198)
> at com.oracle.svm.hosted.FallbackFeature.afterAnalysis(FallbackFeature.java:278)
> at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$9(NativeImageGenerator.java:724)
> at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:63)
> at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:724)
> at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:530)
> at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:445)
> at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
> at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
> at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
> Build on Server(pid: 83895, port: 55676)
> [hellonrx:83895] classlist: 133.91 ms
> [hellonrx:83895] (cap): 1,245.86 ms
> [hellonrx:83895] setup: 1,444.00 ms
> [hellonrx:83895] (typeflow): 2,156.45 ms
> [hellonrx:83895] (objects): 2,398.44 ms
> [hellonrx:83895] (features): 94.27 ms
> [hellonrx:83895] analysis: 4,713.77 ms
> [hellonrx:83895] (clinit): 77.74 ms
> [hellonrx:83895] universe: 226.43 ms
> [hellonrx:83895] (parse): 208.42 ms
> [hellonrx:83895] (inline): 657.30 ms
> [hellonrx:83895] (compile): 1,433.88 ms
> [hellonrx:83895] compile: 2,481.49 ms
> [hellonrx:83895] image: 250.00 ms
> [hellonrx:83895] write: 144.30 ms
> [hellonrx:83895] [total]: 9,472.80 ms
> Warning: Image 'hellonrx' is a fallback image that requires a JDK for execution (use --no-fallback to suppress fallback image generation).
>
> when I do:
>
> System.out.println(“hello, world”)
>
> which I also can do in NetRexx, it works:
>
> ➜ test git:(master) ✗ native-image helloworld -H:+ReportExceptionStackTraces
> Build on Server(pid: 83895, port: 55676)
> [helloworld:83895] classlist: 199.68 ms
> [helloworld:83895] (cap): 1,753.73 ms
> [helloworld:83895] setup: 2,023.70 ms
> [helloworld:83895] (typeflow): 2,465.69 ms
> [helloworld:83895] (objects): 2,636.65 ms
> [helloworld:83895] (features): 128.45 ms
> [helloworld:83895] analysis: 5,309.33 ms
> [helloworld:83895] (clinit): 90.17 ms
> [helloworld:83895] universe: 342.11 ms
> [helloworld:83895] (parse): 317.45 ms
> [helloworld:83895] (inline): 857.58 ms
> [helloworld:83895] (compile): 1,313.64 ms
> [helloworld:83895] compile: 2,665.60 ms
> [helloworld:83895] image: 278.61 ms
> [helloworld:83895] write: 221.28 ms
> [helloworld:83895] [total]: 11,150.81 ms
>
> which makes an awesomely fast hello world.
>
> In short, would it be hard to have the native-image program some understandable error messages? I am stuck for now.
>
> best regards,
>
> René Jansen.
>
>> On 27 Jan 2020, at 03:49, René Jansen <rvjansen at xs4all.nl <mailto:rvjansen at xs4all.nl>> wrote:
>>
>> Hi All, I am trying to get some pointers here to help my understanding. I am the current maintainer of the NetRexx translator - see netrexx.org <https://urldefense.proofpoint.com/v2/url?u=http-3A__netrexx.org_&d=DwMFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=CUkXBxBNT_D5N6HMJ5T9Z6rmvNKYsqupcbk72K0lcoQ&m=-GKRD14OV8ZzxoOlXftJmlGgeW2myEevSjiD0IaM8hM&s=6QeIfZhXhPF2lrweXfUAJPmOjySlkkEojZ2T94VHvX8&e=> - and I am struggling with some concepts. NetRexx is a translator and interpreter that either interprets NetRexx or compiles it to class files, with generated java source as an intermediate. It was thoroughly broken by Java 9 and its modules - being from 1996, NetRexx scans all jars and zips on the classpath in order to make decisions over linking which methods for which classes. It cannot find stuff in modules at the moment.
>>
>> I need to fix this, of course, but I first have implemented some stopgaps to keep NetRexx usable on current JVMs. The first is a Docker image that contains everything to compile classes. My demography, most are older mainframe programmers, makes this rather hard to support, and some refuse outright to try it. Plan B was the GraalVM and a native image for the compiler, which, to my great relief, works! That is, it works for me, but I do not want to release before I understand it better than I do now.
>>
>> I have made native images for the needed entrypoints. The native-image process complains that not everything is supported and generates fallback images, which need a JVM to back it up. This is fine with me, because making programs in NetRexx entails using Java classlibraries from different sources - one of the main advantages of NetRexx is the very well working Java integration.
>>
>> Then everything works as designed. I can install a JDK 13 and run NetRexx, which thinks it runs on Java 8 and can suddenly find all needed signatures. But, now comes the but:
>> I cannot move this executable without moving the NetRexxC.jar file in a corresponding move with it. If you want to try: git clone git://git.code.sf.net/p/netrexx/code <git://git.code.sf.net/p/netrexx/code> , then run make. This works everywhere and even on a JRE because we include the Eclipse compiler. Well, everywhere that is Java 8.
>>
>> Now there are some jars in build/lib. These are apparently needed for execution, and if I move the nrc executable to ~/bin , I have to make a ~/bin/build/lib/NetRexxC.jar for it to find, otherwise it will not find the org.netrexx.process.NetRexxC class, which is the entrypoint for the translator. Moreover, I thought it prudent to change the compiler logo line, to make sure I know what I am running, the native executable or the java class. Now I found out that without regenerating the native executable, the changes (in the org.netrexx.rocess.NrVersion class) were picked up. So when starting a native ‘se;lf-contained’ executable, I am actually branching quite fast to a class in a jar.
>>
>> I have tried reading the documentation. It might be me, but I could not quickly find the technical details that explain this. So to recap:
>>
>> 1) what is actually executed from a native-image generated executable fallback image. Not my classes?
>> 2) how to go about generating a fully self-contained image - that error message does not exactly pinpoint (at all) what the problem is doing this;
>> 3) can I somehow list the symbols in the executable - I did an objdump but I see generic C-library functions and none of my own classes - but a System.getProperty(“java.version”) tells me 8 - while running on a Java 13 JVM.
>> 4) how can the runtime know it is a native executable and no class running in a conventional JVM - this would probably be enough for me to differentiate
>> 5) I see one can make shared object libraries. That would be great, because I have a few entrypoints (nrc, pipe, pipc, nrws) that all uses the same classes. But how to use this? Is there an example?
>> 6) (bonus points) I miss an aarch64 native-image version of JVM8 - is there any chance of this? I am fine with 64-bit only and Raspbian should catch up quickly, but NetRexx on Ubuntu on the Raspi 4 with Java 11 would be great - need a Graal native-image 8 for that. Or do I?
>>
>> thank you very much in advance,
>>
>> best regards,
>>
>> René Jansen.
>
> _______________________________________________
> GraalVM-Users mailing list
> GraalVM-Users at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/graalvm-users
> On 28 Jan 2020, at 15:21, Julia Kindelsberger <JULIA.KINDELSBERGER at ORACLE.COM> wrote:
>
> Hi Rene,
>
> Thank you. It would help if you share your issue here after creating it.
>
> Best Regards,
> Julia
>
>> On 28 Jan 2020, at 14:10, René Jansen <rvjansen at xs4all.nl> wrote:
>>
>> Hi Julia,
>>
>> will do - did not know this was the preferred way.
>>
>> Thanks for your reply.
>>
>> best regards,
>>
>> René
>>
>>> On 28 Jan 2020, at 13:44, Julia Kindelsberger <JULIA.KINDELSBERGER at ORACLE.COM <mailto:JULIA.KINDELSBERGER at ORACLE.COM>> wrote:
>>>
>>> Hi Rene,
>>>
>>> We are sorry for the delayed reply.
>>>
>>> Could you please post this on a github issue (https://github.com/oracle/graal <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_oracle_graal&d=DwMFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=LndIBHZ7QYtPXGbQUL7va3Uolt9OktqTU0ozohHqBEY&m=WPdGPaOPfeOUcxmFB7B0_5Jh92LCjeVLpQGEjxKwrjM&s=NULp6w2nsCtOJvRH_E4u98qhXXyegD3hJD5GdtP25T4&e=>)? We can then take it from there.
>>>
>>
> _______________________________________________
> GraalVM-Users mailing list
> GraalVM-Users at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/graalvm-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/graalvm-users/attachments/20200128/6bb926ed/attachment-0001.html
More information about the GraalVM-Users
mailing list