Java Curation: Difference between revisions

From Flashpoint Datahub
Jump to navigation Jump to search
m (Updated FlashpointDisableFocusTraversalKeysWrapperApplet's broken download link.)
(Updated page for Core 5.)
Line 1: Line 1:
This guide will show you how to curate Java games and animations using Flashpoint Core, as well as dealing with any problems that may come up in the process.  
This guide will show you how to curate Java games and animations using Flashpoint Core, as well as dealing with any problems that may come up in the process.


Note that Flashpoint doesn't currently support Java Web Start or Microsoft Java.  
Note that Flashpoint doesn't currently support Java Web Start or Microsoft Java.  


As of Flashpoint Core 4, Java applets in the browser are now supported. More information about how to use this functionality is coming soon!
This page is accurate as of '''[https://bluemaxima.org/flashpoint/downloads/ Flashpoint Core 5]'''.


== How to recognize them in the wild ==
== Recognizing Java applets ==


Unlike other technologies such as Flash or Shockwave, Java applets are most commonly embedded on a page using an applet tag rather than an object/embed tag. Finding these tags is important since they're necessary to get the games running in Flashpoint.
Unlike other technologies such as Flash or Shockwave, Java applets are most commonly embedded on a page using an applet tag rather than an object/embed tag. Finding these tags is important since they're necessary to get the games running in Flashpoint.
Line 28: Line 28:
</pre>
</pre>


Java Applets may also be embedded using the object and embed tags, although this appears to be less common:
Java applets may also be embedded using the object and embed tags, although this appears to be less common:


<pre>
<pre>
Line 45: Line 45:
== Curation Steps ==
== Curation Steps ==


Flashpoint runs Java applets offline by using an application called the AppletViewer. You can think of it like the equivalent to a Flash projector. Let's take a look at a simple example by curating the game Realspace 2 - Emperor's Revenge from https://www.miniclip.com/games/real-space-2/en/
Flashpoint runs Java applets offline by using an application called the AppletViewer. You can think of it like the equivalent to a Flash projector. Let's take a look at a simple example by curating the game "Realspace 2 - Emperor's Revenge" from https://www.miniclip.com/games/real-space-2/en/


<ol>
<ol>
Line 84: Line 84:
However, for this and most other Java games, the archive only has the game's code. We still need to download the rest of the assets. This can be done in several ways, including:
However, for this and most other Java games, the archive only has the game's code. We still need to download the rest of the assets. This can be done in several ways, including:


* Checking what the game is requesting by running it, looking at the Redirector/Fiddler and downloading these files manually.
* Checking what the game is requesting by running it, looking at the '''Logs''' tab in the Flashpoint launcher, and downloading these files manually.
* Using a tool like wget or [[Helpful Tools#Software for Archiving|cURLsDownloader]] to download the assets from specific URLs. This is especially useful if you want to download the files from the Wayback Machine.
* Using a tool that automatically downloads these files from the original site as they're requested by the game. See the [[MAD4FP]] page for more details on how to do this.
* Using a tool that automatically downloads these files from the original site as they're requested by the game. See the [[MAD4FP]] page for more details on how to do this.
</li>
</li>
Line 98: Line 99:
</pre>
</pre>


The platform and application path should be the same for all Java curations. Make sure that the launch command starts with http:// and '''not''' https://
The platform should be the same for all Java curations. Make sure that the launch command starts with http:// and '''not''' https://


You can run the game by double-clicking its thumbnail in the launcher. You should now see the game running in the AppletViewer:
You can run the game by double-clicking its thumbnail in the launcher. You should now see the game running in the AppletViewer:
Line 112: Line 113:
== Curation Tips ==
== Curation Tips ==


* You may sometimes see a parameter in the applet tag called "cabbase" that specifies a CAB file. This was used to allow the same applet to run in different browsers - Netscape would download the JAR or ZIP in the archive attribute while Internet Explorer would download the CAB in the cabbase parameter. Although the AppletViewer doesn't request this file, you should still include it in your submission for preservation's sake.
* You may sometimes see a parameter in the applet tag called <code>cabbase</code> that specifies a CAB file. This was used to allow the same applet to run in different browsers - Netscape would download the JAR or ZIP in the archive attribute while Internet Explorer would download the CAB in the cabbase parameter. Although the AppletViewer doesn't request this file, you should still include it in your submission for preservation's sake.


* The AppletViewer ignores everything in the file it reads except for the tags that embed a Java applet. You can see the tag that was read by the AppletViewer by going to <code>Applet > Tag...</code>
* The AppletViewer ignores everything in the file it reads except for the tags that embed a Java applet. You can see the tag that was read by the AppletViewer by going to <code>Applet > Tag...</code>


* While running in the AppletViewer, games won't be able to call JavaScript methods or interact with other HTML elements. Some games may rely on calling JavaScript methods to save their progress in the browser's cookies, like Need for Madness or Insaniquarium. You should leave an appropriate note in the metadata if the game doesn't save or load its progress properly. Applets that call JavaScript methods using the JSObject class will always throw the following exception: <code>netscape.javascript.JSException: Unexpected error: This method should not be used unless loaded from plugin.jar</code>.
* You can sometimes find more information about the author, version and copyright of an applet by going to <code>Applet > Info...</code> in the AppletViewer.
 
* If a Java game requests external assets during gameplay and you don't have time to play through the entire game, try opening the game's <code>.class</code> files in a hex editor such as [[Helpful Tools#General Software|HxD]. References to external assets are often plain text strings, so try searching for common asset types such as <code>.gif</code> and <code>.jpg</code> images or <code>.au</code> sounds. Make a list of these assets, then download them from the internet and add them to the game's directory.
 
== Application Paths ==
 
This section will list all the possible values for the <code>Application Path</code> field in the metadata and launcher. Note that both of these currently use Java 8 update 181. If you encounter any problems that you think might be solved by using an older Java version, read the steps in the [[#Troubleshooting|troubleshooting section]] first.
 
=== startJava.bat ===
 
Use <code>FPSoftware\startJava.bat</code> to run the applet in the AppletViewer (as we saw in [[#Curation Steps|the example above]]). If you use this application path, put the URL to the file that contains the <code><applet></code> tag in the <code>Launch Command</code> field. For example:
 
<pre>
Application Path: FPSoftware\startJava.bat
Launch Command: http://www.example.com/game/index.html
</pre>
 
If the URL contains spaces, replace them with <code>%20</code>. For example:
<pre>
Launch Command: http://www.example.com/game/index%20with%20spaces.html
</pre>
 
You can also pass extra command line arguments to the Java Virtual Machine (JVM) by placing them '''after''' the URL. These options must be prefixed with '''-J''' For example:
<pre>
Launch Command: http://www.example.com/game/index.html -J-Duser.language=en -J-Duser.country=US
</pre>
Read the [[#Troubleshooting|troubleshooting section]] to see how these may be used to solve some common problems.
 
If the URL contains an ampersand (&) character, wrap it in double quotes. This happens when you use PHP to set certain parameters dynamically in the applet tag. For example:
<pre>
Launch Command: "http://www.example.com/game/index.php?param1=value1&param2=value2"
</pre>
Where "value1" and "value2" could change what the parameters are set to.
 
=== startJavaInBrowser.bat ===
 
Use <code>FPSoftware\startJavaInBrowser.bat</code> to run the applet in a browser. Unlike the application path mentioned above, you don't have to necessarily put the URL to the file that contains the <code><applet></code> tag in the <code>Launch Command</code> field. For example:
 
<pre>
Application Path: FPSoftware\startJavaInBrowser.bat
Launch Command: http://www.example.com/game/index.html
</pre>


* You can pass options to the Java Virtual Machine by putting -J before each one. These options must come '''before''' the launch command. For example, you can use <code><nowiki>-J-XshowSettings http://www.example.com/game.html</nowiki></code> to show all Java settings, like system properties, available locales, etc.
The previous notes about the launch commands also apply, with the exception of the extra command line arguments. If you use <code>startJavaInBrowser.bat</code>, these options are '''not''' prefixed with '''-J'''. For example:
<pre>
Launch Command: http://www.example.com/game/index.html -Duser.language=en -Duser.country=US
</pre>


* You can sometimes find more information about the author, version and copyright of an applet by going to <code>Applet > Info...</code> in the AppletViewer.
In addition to this, you '''must''' always follow the next two steps. These will ensure that the applet is allowed to run in its domain with the correct Java version.
 
<ol>
 
<li>
Add the game's domain to the end of the exception site list file in <code>FPSoftware\Java\JDK_1.8.0_181\jre\lib\exception.sites</code>. For this example, this would be <code>http://www.example.com/</code>. If you're curating multiple games from the same domain, you only have to do this once. Be sure to include this information in the <code>Curation Notes</code> field in the metadata. For example: <code>Before testing the game, add http://www.example.com/ to the exception site list file in FPSoftware\Java\JDK_1.8.0_181\jre\lib\exception.sites".</code>
</li>
 
<li>
Add these two parameters to the <code><applet></code> tag:
<pre>
<param name="java_version" value="1.8.0_181">
<param name="separate_jvm" value="true">
</pre>
If the applet uses an <code><embed></code> tag instead, these parameters would look like:
<pre>
<embed java_version="1.8.0_181" separate_jvm="true">
</pre>
These parameters have no effect if the same applet is launched in the AppletViewer via <code>startJava.bat</code>.
</li>
 
</ol>
 
Every applet that runs in a browser will show a security prompt that must be accepted before starting. To make this clear to Flashpoint users, add the following to the beginning of the <code>Game Notes</code> field in the metadata:
<pre>
This Java game will run in a browser. Make sure to accept any security prompts that show up while playing.
</pre>
 
<gallery mode="packed">
 
Image:java_browser_curation_blocked_example.png|The security prompt that shows up if the applet's domain isn't added to the exception site list.
 
Image:java_browser_curation_warning_example.png|The security prompt that asks the user to allow the applet to run.
 
Image:java_browser_curation_certificate_warning_example.png|Another security prompt that asks the user to allow the applet to run. This one only appears if the applet includes an expired certificate.
 
Image:java_browser_curation_example.png|The game Urbanoids running in a browser. Notice the various page elements that are used to interact with the game, and the three music tracks (MIDI, MOD, SID) that each require different plugins to be played.
 
</gallery>
 
Since in-browser Java applets are currently only supported on Windows, you should provide an option to launch them in the AppletViewer (if possible). You can do so by making an additional application called "Play in the AppletViewer" with the <code>startJava.bat</code> application path, and by adding the following to the <code>Game Notes</code> field in the metadata:
<pre>
If playing in a browser doesn't work, try using the "Play in the AppletViewer" option under Additional Applications.
</pre>
 
==== When to run applets in a browser ====
 
You should run Java applets in a browser if any of the following conditions are met:
* The page has two or more applets that communicate via JavaScript. Note that it's possible for multiple applets to communicate with each other using the AppletViewer, provided they do so using the [https://docs.oracle.com/javase/8/docs/api/java/applet/AppletContext.html#getApplet-java.lang.String- AppletContext.getApplet] method.
* The applet relies on page element interaction (e.g. pressing buttons) to be played.
* The applet uses browser cookies to save the game's progress. These are saved and loaded using JavaScript.
* The page contains other multimedia elements that enhance the experience. For example, the page might embed Flash or Shockwave movies, or play music (MIDI, MOD).
These aren't hard requirements, you can curate any applet using <code>startJavaInBrowser.bat</code> if you feel like it captures the original experience. If you decide to curate an applet using this application path, you should also include any other assets (images, sounds) that are embedded on the same page.
 
==== Browser curation tips ====
 
If some JavaScript code on the page is meant to interact with the applet, you should check how it's doing so. Older applets may rely on old JavaScript functionalities that modern browsers don't support. For examples, in the game Urbanoids, the player has to click a button on the page to start the game:
<pre>
function start_it() {
document.game.clickStart();
}
 
<applet code="noids.class" width=240 height=263 name="game">
<param name="java_version" value="1.8.0_181">
<param name="separate_jvm" value="true">
</applet>
</pre>
This wouldn't work because the applet tag uses the attribute <code>name</code>. To fix this, we would need to change it to <code>id</code>:
<pre>
<applet code="noids.class" width=240 height=263 id="game">
</pre>
 
== Debugging Java applets ==
 
If an applet doesn't seem to be working properly, check if any error occurs while it's running. You can see what the applet writes to the standard output and error output by editing the batch files specified in the Application Path. It's highly recommended that you enable these features when curating Java applets.
 
=== startJava.bat ===


* If a Java game requests external assets during gameplay and you don't have time to play through the entire game, try opening the game's <code>.class</code> files in a hex editor such as HxD. References to external assets are often plain text strings, so try searching for common asset types such as <code>.gif</code> and <code>.jpg</code> images or <code>.au</code> sounds. Make a list of these assets, then download them from the internet and add them to the game's directory.
Open <code>FPSoftware\startJava.bat</code> and change <code>"java"</code> to <code>"javadebug"</code> in the following line:
<pre>
FlashpointSecurePlayer -n "java" -sw %*
</pre>


== Debugging ==
A separate window containing the applet's output will now show up everytime you run the AppletViewer.


You can see what the applet writes to the standard output and error output by replacing <code>>NUL</code> in the following line in <code>FPSoftware\startJava.bat</code>:
=== startJavaInBrowser.bat ===


<code>CALL JDKPortable\CommonFiles\JDK\bin\appletviewer.exe -J-Dhttp.proxyHost=127.0.0.1 -J-Dhttp.proxyPort=8888 %* >NUL 2>&1</code>
Open <code>FPSoftware\startJava.bat</code> and change <code>OFF</code> to <code>ON</code> in the following line:
<pre>
SET FLASHPOINT_DEBUG_MODE=OFF
</pre>


<code>>NUL</code> normally suppresses the output, but you can also:
This will always show the Java console when the applet runs and will place a Java icon in the system tray, which you can use to bring up the console at any time.
* Remove it to have the output show up in the Flashpoint Launcher's Logs tab.
* Replace it with <code>>"core_java.log"</code> to write the output to that file. In this case, it would be located in <code>FPSoftware\core_java.log</code>.


== Useful Tools and Resources ==
== Useful Tools and Resources ==
Line 142: Line 267:
* Decompilers: [https://github.com/java-decompiler/jd-gui/releases JD-GUI], [https://github.com/deathmarine/Luyten/releases Luyten]
* Decompilers: [https://github.com/java-decompiler/jd-gui/releases JD-GUI], [https://github.com/deathmarine/Luyten/releases Luyten]


* Bytecode Editors: [http://dirty-joe.com/ dirtyJOE], [https://github.com/GraxCode/JByteMod-Beta/releases JByteMod]
* Bytecode Editors: [http://dirty-joe.com/ dirtyJOE], [https://github.com/GraxCode/JByteMod-Beta/releases JByteMod]. Always check if there's a simpler way to get Java applets working. Only modify the original files as a last resort!


* Troubleshooting Guides for [https://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-Desktop/html/toc.html Java SE 7] and [https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/index.html Java SE 8].
* API Specification for [https://docs.oracle.com/javase/8/docs/api/ Java SE 8].


See also: [[Helpful Tools]].
See also: [[Helpful Tools]].
Line 150: Line 275:
== Troubleshooting ==
== Troubleshooting ==


This section will cover some of the problems that may arise when curating Java games and animations. If you're experiencing an issue not mentioned in this section be sure to ask for help in the <code>#curator-lounge</code> channel in the Flashpoint Discord server.
This section will cover some of the problems that may arise when curating Java games and animations. It's highly recommended that you enable the debugging options mentioned above when curating Java applets. If you're experiencing an issue not mentioned in this section be sure to ask for help in the <code>#curator-lounge</code> channel in the Flashpoint Discord server.


=== The applet closes immediately when I try to run it ===
=== The applet closes immediately when I try to run it ===
Line 162: Line 287:
=== The applet isn't displaying certain characters correctly ===
=== The applet isn't displaying certain characters correctly ===


Try setting the correct character encoding and locale properties using the -J option in the AppletViewer. For example, if a game from GeoCities Japan needed to display Japanese characters, you would add all of the following '''before''' the game's launch command: <code>-J-Dfile.encoding=UTF8 -J-Duser.language=ja -J-Duser.country=JP</code>.
Try setting the correct character encoding and locale properties in the AppletViewer. For example, if a game from GeoCities Japan needed to display Japanese characters, you would add '''all''' of the following to the end of the game's launch command: <code>-J-Dfile.encoding=UTF8 -J-Duser.language=ja -J-Duser.country=JP</code>. If you're running the applet in a browser with <code>startJavaInBrowser.bat</code>, remove the '''-J''' prefix.


[[File:java_character_encoding_comparision.png|center|thumb|A comparision between using the English locale (left) and the Japanese locale (right)]]
[[File:java_character_encoding_comparision.png|center|thumb|A comparision between using the English locale (left) and the Japanese locale (right)]]


You can also see what the current character encoding and locale settings are by using <code>-J-XshowSettings</code>.
You can also see what the current character encoding and locale settings are in the AppletViewer by using <code>-J-XshowSettings</code>. If you're running the applet in a browser with <code>startJavaInBrowser.bat</code>, don't use this option. Instead, [[#startJavaInBrowser.bat_2 | enable the Java console]] and press 's' to show the current settings.


If you would like to know more about which character encodings and locales are supported in Java 8, see:
If you would like to know more about which character encodings and locales are supported in Java 8, see:
Line 181: Line 306:
# Change the code attribute to <code>FlashpointDisableFocusTraversalKeysWrapperApplet</code>. The wrapper applet will run before loading the main one.
# Change the code attribute to <code>FlashpointDisableFocusTraversalKeysWrapperApplet</code>. The wrapper applet will run before loading the main one.


This wrapper disables all forward and backwards focus traversal keys (like Tab and Shift-Tab), so that these key events become available to the key event listeners. These keys would normally be used to change the keyboard focus in components like buttons and text fields.
This wrapper disables all forward and backwards focus traversal keys (like Tab and Shift-Tab), so that these key events become available to the key event listeners. These keys would normally be used to change the keyboard focus in components like buttons and text fields. Applets affected by this problem expect the Tab key's input to work since they were made before Java 1.4 (2002) where this change was introduced.


Here's an example of a modified applet tag from the game Teslatron, where the second player uses the Tab key to drop bombs:
Here's an example of a modified applet tag from the game Teslatron, where the second player uses the Tab key to drop bombs:
Line 201: Line 326:
* https://www.developer.com/java/other/article.php/2244591/Changing-Focus-Traversal-Keys-in-Java-V14.htm
* https://www.developer.com/java/other/article.php/2244591/Changing-Focus-Traversal-Keys-in-Java-V14.htm


=== The applet doesn't start and shows the following error: java.lang.ClassFormatError: Incompatible magic value 1008813135 in class file ===
=== The applet doesn't start in the AppletViewer and shows the following error: netscape.javascript.JSException ===


This error means that the class file is in fact just an HTML file. This is likely due to being served a 404 page because the class file is missing. Try opening the file in a text editor or browser to confirm this. The value 1008813135 represents "<!DO", in other words, the start of an HTML <!DOCTYPE> declaration. You may see a different magic value in the error depending on the wrong type of file you were served. The class file format is always supposed to have 3405691582 (0xCAFEBABE in hexadecimal) as the first four bytes.
This means that the applet wants to interface with the browser, likely trying to execute some JavaScript code. This error may be ignored in some cases. For example, an applet that uses JavaScript to track the number of times the game has been played. If this doesn't impact the gameplay or show any side effects, you can safetly ignore it. Otherwise, you will need to run the applet in a browser. Refer to this section to learn how to do it.


=== The applet doesn't start and shows the following error: java.lang.ClassFormatError: Name index 0 in LocalVariableTable has bad constant type in class file ===
=== The applet doesn't start and shows the following error: java.lang.ClassFormatError: Incompatible magic value 1008813135 in class file ===
 
When [https://docs.oracle.com/javase/8/docs/technotes/tools/windows/javac.html compiling Java source files], you can specify certain debug options, one of which generates information about the local variables of each method (the LocalVariableTable attribute). This error happens when there's a single LocalVariableTable that is invalid according to the [https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.13 class file format]. It seems to be most common with applets made around 1997-2003. Although it might have been possible to run the applet in older Java versions, we'll have to remove this offending LocalVariableTable in order to run it in Flashpoint. To do this, we'll use a Java class file editor called [http://dirty-joe.com/ dirtyJOE].
 
[[File:java_dirtyjoe_localvariabletable_comparision.png|center|thumb|A comparision between a normal LocalVariableTable (left) and an invalid one (right) using dirtyJOE. Notice the use of an [https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.3.2 invalid descriptor] (NULL) in this last one.]]
 
Before starting, you should make a copy of the class files so that your submission includes both the original and the modified files. On Windows, you can make a copy of the original class files in the current directory by running the following on the Command Prompt: <code>FOR /F "delims=" %c IN ('DIR /B *.class') DO (COPY /B "%c" "original_%c")</code>.


If the class files are packaged in JAR or ZIP archives, decompress them. After following the next steps and fixing this error, compress them again and rename the archives (by adding "_hacked" to the end, for example). This way both the original and modified archives are included. Be sure to update the archive attribute in the applet tag so that the modified files are loaded instead of the original ones.
This error means that the class file is in fact just an HTML file. This is likely due to being served a 404 page because the class file is missing. Try opening the file in a text editor or browser to confirm this. The value 1008813135 represents <code><!DO</code>, in other words, the start of an HTML <code><!DOCTYPE></code> declaration. You may see a different magic value in the error depending on the wrong type of file you were served. The class file format is always supposed to have 3405691582 (0xCAFEBABE in hexadecimal) as the first four bytes.


To fix this error, follow the next steps for each class file:
=== The applet doesn't start and shows one of the following error: java.lang.ClassFormatError: Name index 0 in LocalVariableTable has bad constant type in class file ===
# Run dirtyJOE and open the class file.
# Click on the Methods tab to see the list of all the methods defined in this class.
# Start at the top of the list and find the first method that shows a Code attribute in the Attributes panel. In the majority of the cases, it will be the first method. Note that it's possible for the class file to be fine, even if the applet's other class files suffer from this problem. Only modify files that prevent the applet from running.
# Double-click on the Code attribute. You should now see a new window titled Code Editor followed by that method's name.
# In the Attributes panel of this new window, right-click on the LocalVariableTable attribute and select "Delete Attribute".
# Close the Code Editor window and save the modified class file by pressing Ctrl + S or by going to File > Save. Do not change the class file's name.


[[File:java_dirtyjoe_localvariabletable_steps.png|center|thumb|A visual guide for the steps mentioned above. Note that the numbers don't match up.]]
This error means that Java encountered a problem while checking if the class file is formatted correctly. You can fix this by adding the following option to the end of the launch command: <code>-J-Xverify:none</code>. If you're running the applet in a browser with <code>startJavaInBrowser.bat</code>, use <code>-Xverify:none</code> instead.


Before submitting your curation, make sure that the metadata says that the game was "Hacked" (see the [[Curation Format]] page for more details) and that the notes mention why. For example, you could add one of the following to the beginning of the notes field:
This command line argument tells Java to skip the bytecode verification step, meaning this error will be ignored and the applet can continue running. Note that this should be done sparingly, and only when you know it's an error of this nature. You should always check if the applet works correctly after using it.


Hacked to fix an issue in every .class file that prevents the game from running. The original unhacked files can be found next to the hacked ones with their names prefixed with "original_".
This same fix also applies to the following errors: <code>java.lang.ClassFormatError: Invalid start_pc 65535 in LocalVariableTable in class file</code> and <code>java.lang.ClassFormatError: Illegal field name "?" in class</code>.


or
More information about this error: when [https://docs.oracle.com/javase/8/docs/technotes/tools/windows/javac.html compiling Java source files], you can specify certain debug options, one of which generates information about the local variables of each method (the LocalVariableTable attribute). This error happens when there's a single LocalVariableTable that is invalid according to the [https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.13 class file format]. It seems to be most common with applets in the late 1990s and early 2000s which were compiled using older Java versions.


Hacked to fix an issue in every .class file that prevents the game from running. The original unhacked files can be found in game.zip.
[[File:java_dirtyjoe_localvariabletable_comparision.png|center|thumb|A comparision between a normal LocalVariableTable (left) and an invalid one (right) using [http://dirty-joe.com/ dirtyJOE]. Notice the use of an [https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.3.2 invalid descriptor] (NULL) in this last one.]]


== Developer Specific Troubleshooting ==
== Developer Specific Troubleshooting ==
Line 243: Line 356:
Note that you won't experience this issue if this is already your default locale. To avoid curating a game that might crash for another player, try playing the game with the following locale settings: <code>-J-Duser.language=es -J-Duser.country=ES</code>. You can see what the current locale settings are by using <code>-J-XshowSettings</code>.
Note that you won't experience this issue if this is already your default locale. To avoid curating a game that might crash for another player, try playing the game with the following locale settings: <code>-J-Duser.language=es -J-Duser.country=ES</code>. You can see what the current locale settings are by using <code>-J-XshowSettings</code>.


Why this happens: the game uses [https://docs.oracle.com/javase/8/docs/api/java/text/NumberFormat.html#getInstance-- <code>NumberFormat.getInstance()</code>] to get a number format to format the player's score. This method uses the default locale, meaning that a number like one thousand is formatted as either 1,000 or 1.000 [https://docs.oracle.com/cd/E19455-01/806-0169/overview-9/index.html depending on your locale]. The game's score font data is only defined for the numbers 0 through 9 and the comma, but not for the period. While the game is iterating over the formatted score string, the image data relative to the period will be returned as null, causing the game to eventually throw a NullPointerException when it accesses the image's attributes.
Why this happens: the game uses [https://docs.oracle.com/javase/8/docs/api/java/text/NumberFormat.html#getInstance-- <code>NumberFormat.getInstance()</code>] to get a number format to format the player's score. This method uses the default locale, meaning that a number like one thousand is formatted as either <code>1,000</code> or <code>1.000</code> [https://docs.oracle.com/cd/E19455-01/806-0169/overview-9/index.html depending on your locale]. The game's score font data is only defined for the numbers 0 through 9 and the comma, but not for the period. While the game is iterating over the formatted score string, the image data relative to the period will be returned as null, causing the game to eventually throw a NullPointerException when it accesses the image's attributes.


=== GameHouse ===
=== GameHouse ===

Revision as of 01:58, 21 May 2020

This guide will show you how to curate Java games and animations using Flashpoint Core, as well as dealing with any problems that may come up in the process.

Note that Flashpoint doesn't currently support Java Web Start or Microsoft Java.

This page is accurate as of Flashpoint Core 5.

Recognizing Java applets

Unlike other technologies such as Flash or Shockwave, Java applets are most commonly embedded on a page using an applet tag rather than an object/embed tag. Finding these tags is important since they're necessary to get the games running in Flashpoint.

Here's an example of the most basic type you can find:

<applet code="iceblox.class" width="390" height="350"></applet>

Here's a more complex example that shows certain parameters being passed to the game and that the class files may be packaged in a JAR/ZIP/CAB archive. If you tried to run this game on a browser that didn't support Java, you'd only see "Unfortunately, your browser does not support Java". This can be a good way to know if the web game or animation you're trying to curate uses Java.

<applet code="gamehouse.SuperApplet.class" archive="Collapse.zip" height="400" width="460" viewastext mayscript>
	<param name="cabbase" value="Collapse.cab">
	<param name="canvas" value="Collapse.CollapseCanvas">
	<param name="bgcolor" value="FFFFFF">
	<param name="textcolor" value="000000">
	<param name="buyurl" value="http://get.games.yahoo.com/proddesc?gamekey=spcollapse">
	<param name="target" value="_blank">

	Unfortunately, your browser does not support Java.
</applet>

Java applets may also be embedded using the object and embed tags, although this appears to be less common:

<object classid="clsid:8AD9C840-044E-11D1-B3E9-00805F499D93" codebase="http://java.sun.com/products/plugin/1.2/jinstall-12-win32.cab#Version=1,2,0,0" width="100%" height="100%" name="Rockz" alt="Java applet failure">
	<param name="code" value="rockz.Run.class" >
	<param name="archive" value="rockz.jar" >
	<param name="name" value="Rockz" >
	<param name="type" value="application/x-java-applet;version=1.2">

	<embed code="rockz.Run.class" archive="rockz.jar" name="Rockz" type="application/x-java-applet;version=1.2" width="100%" height="100%" alt="Java applet failure" pluginspage="http://java.sun.com/products/plugin/1.2/plugin-install.html"></embed>
</object>

To find these tags, right-click the page with the applet, click "Inspect Element" or "View Page Source" and then search for "applet".

Curation Steps

Flashpoint runs Java applets offline by using an application called the AppletViewer. You can think of it like the equivalent to a Flash projector. Let's take a look at a simple example by curating the game "Realspace 2 - Emperor's Revenge" from https://www.miniclip.com/games/real-space-2/en/

  1. Open this page in your browser. You should see the text saying that Java is not enabled or installed. This indicates that it's a Java game, so we should take a look at the page's source and try to find the applet tag. If we search for "applet", we'll find the following:
    <applet width="500" height="500" code="gameapp.class" archive="gameapp.jar">
    	<param name="SIZE_SCALE" value="1.0" />
    	If you see this message you do not have Java enabled or do not have Java installed.</br><a href="http://wwww.java.com">Please see Java.com</a> to download the latest version of Java.</a>
    </applet>
    

    Note that the page could have also written the tag dynamically which means that you wouldn't be able to find it by just checking its source. In this case you would use your browser's "Inspect Element" option and search for it there.

  2. In your Flashpoint Core directory, navigate to Server\htdocs. In htdocs, create a series of directories that match the original site's structure. In this case, we'd create four directories, each inside the previous one: www.miniclip.com\games\real-space-2\en\
  3. Navigate to the last directory (in this case \en\) and create the file that will contain the applet tag we found in step 1. You should give it the same name as the original site. In this case there isn't one, so we'll just create a file called index.html.
  4. Open this file you've just created in a text editor and paste the applet tag in there.
    <applet width="500" height="500" code="gameapp.class" archive="gameapp.jar">
    	<param name="SIZE_SCALE" value="1.0" />
    </applet>
    
  5. Download the game's assets (code, images, sounds, etc) and put them in the correct directories so that they also match the original site's structure. It's possible that all the files needed are inside the archive (called gameapp.jar here). If they were and we didn't need any additional files then we could move onto the next step. However, for this and most other Java games, the archive only has the game's code. We still need to download the rest of the assets. This can be done in several ways, including:
    • Checking what the game is requesting by running it, looking at the Logs tab in the Flashpoint launcher, and downloading these files manually.
    • Using a tool like wget or cURLsDownloader to download the assets from specific URLs. This is especially useful if you want to download the files from the Wayback Machine.
    • Using a tool that automatically downloads these files from the original site as they're requested by the game. See the MAD4FP page for more details on how to do this.
  6. You can now test your curation in Flashpoint Core. Press the "New Game" button on the bottom right corner and enter the game's title, platform, application path and launch command. In this case, they would be:
    Title: Realspace 2 - Emperor's Revenge
    Platform: Java
    Application Path: FPSoftware\startJava.bat
    Launch Command: http://www.miniclip.com/games/real-space-2/en/index.html
    

    The platform should be the same for all Java curations. Make sure that the launch command starts with http:// and not https://

    You can run the game by double-clicking its thumbnail in the launcher. You should now see the game running in the AppletViewer:

    Realspace 2 - Emperor's Revenge running in the AppletViewer
  7. Create a curation by following the Curation Format. Don't forget to copy the information from the four fields you filled out in the previous step. Then, follow the steps explained in Instructions for Curating to submit the game.

Curation Tips

  • You may sometimes see a parameter in the applet tag called cabbase that specifies a CAB file. This was used to allow the same applet to run in different browsers - Netscape would download the JAR or ZIP in the archive attribute while Internet Explorer would download the CAB in the cabbase parameter. Although the AppletViewer doesn't request this file, you should still include it in your submission for preservation's sake.
  • The AppletViewer ignores everything in the file it reads except for the tags that embed a Java applet. You can see the tag that was read by the AppletViewer by going to Applet > Tag...
  • You can sometimes find more information about the author, version and copyright of an applet by going to Applet > Info... in the AppletViewer.
  • If a Java game requests external assets during gameplay and you don't have time to play through the entire game, try opening the game's .class files in a hex editor such as [[Helpful Tools#General Software|HxD]. References to external assets are often plain text strings, so try searching for common asset types such as .gif and .jpg images or .au sounds. Make a list of these assets, then download them from the internet and add them to the game's directory.

Application Paths

This section will list all the possible values for the Application Path field in the metadata and launcher. Note that both of these currently use Java 8 update 181. If you encounter any problems that you think might be solved by using an older Java version, read the steps in the troubleshooting section first.

startJava.bat

Use FPSoftware\startJava.bat to run the applet in the AppletViewer (as we saw in the example above). If you use this application path, put the URL to the file that contains the <applet> tag in the Launch Command field. For example:

Application Path: FPSoftware\startJava.bat
Launch Command: http://www.example.com/game/index.html

If the URL contains spaces, replace them with %20. For example:

Launch Command: http://www.example.com/game/index%20with%20spaces.html

You can also pass extra command line arguments to the Java Virtual Machine (JVM) by placing them after the URL. These options must be prefixed with -J For example:

Launch Command: http://www.example.com/game/index.html -J-Duser.language=en -J-Duser.country=US

Read the troubleshooting section to see how these may be used to solve some common problems.

If the URL contains an ampersand (&) character, wrap it in double quotes. This happens when you use PHP to set certain parameters dynamically in the applet tag. For example:

Launch Command: "http://www.example.com/game/index.php?param1=value1&param2=value2"

Where "value1" and "value2" could change what the parameters are set to.

startJavaInBrowser.bat

Use FPSoftware\startJavaInBrowser.bat to run the applet in a browser. Unlike the application path mentioned above, you don't have to necessarily put the URL to the file that contains the <applet> tag in the Launch Command field. For example:

Application Path: FPSoftware\startJavaInBrowser.bat
Launch Command: http://www.example.com/game/index.html

The previous notes about the launch commands also apply, with the exception of the extra command line arguments. If you use startJavaInBrowser.bat, these options are not prefixed with -J. For example:

Launch Command: http://www.example.com/game/index.html -Duser.language=en -Duser.country=US

In addition to this, you must always follow the next two steps. These will ensure that the applet is allowed to run in its domain with the correct Java version.

  1. Add the game's domain to the end of the exception site list file in FPSoftware\Java\JDK_1.8.0_181\jre\lib\exception.sites. For this example, this would be http://www.example.com/. If you're curating multiple games from the same domain, you only have to do this once. Be sure to include this information in the Curation Notes field in the metadata. For example: Before testing the game, add http://www.example.com/ to the exception site list file in FPSoftware\Java\JDK_1.8.0_181\jre\lib\exception.sites".
  2. Add these two parameters to the <applet> tag:
    <param name="java_version" value="1.8.0_181">
    <param name="separate_jvm" value="true">
    

    If the applet uses an <embed> tag instead, these parameters would look like:

    <embed java_version="1.8.0_181" separate_jvm="true">
    

    These parameters have no effect if the same applet is launched in the AppletViewer via startJava.bat.

Every applet that runs in a browser will show a security prompt that must be accepted before starting. To make this clear to Flashpoint users, add the following to the beginning of the Game Notes field in the metadata:

This Java game will run in a browser. Make sure to accept any security prompts that show up while playing.

Since in-browser Java applets are currently only supported on Windows, you should provide an option to launch them in the AppletViewer (if possible). You can do so by making an additional application called "Play in the AppletViewer" with the startJava.bat application path, and by adding the following to the Game Notes field in the metadata:

If playing in a browser doesn't work, try using the "Play in the AppletViewer" option under Additional Applications.

When to run applets in a browser

You should run Java applets in a browser if any of the following conditions are met:

  • The page has two or more applets that communicate via JavaScript. Note that it's possible for multiple applets to communicate with each other using the AppletViewer, provided they do so using the AppletContext.getApplet method.
  • The applet relies on page element interaction (e.g. pressing buttons) to be played.
  • The applet uses browser cookies to save the game's progress. These are saved and loaded using JavaScript.
  • The page contains other multimedia elements that enhance the experience. For example, the page might embed Flash or Shockwave movies, or play music (MIDI, MOD).

These aren't hard requirements, you can curate any applet using startJavaInBrowser.bat if you feel like it captures the original experience. If you decide to curate an applet using this application path, you should also include any other assets (images, sounds) that are embedded on the same page.

Browser curation tips

If some JavaScript code on the page is meant to interact with the applet, you should check how it's doing so. Older applets may rely on old JavaScript functionalities that modern browsers don't support. For examples, in the game Urbanoids, the player has to click a button on the page to start the game:

function start_it() {
	document.game.clickStart();
}

<applet code="noids.class" width=240 height=263 name="game">
	<param name="java_version" value="1.8.0_181">
	<param name="separate_jvm" value="true">
</applet>

This wouldn't work because the applet tag uses the attribute name. To fix this, we would need to change it to id:

<applet code="noids.class" width=240 height=263 id="game">

Debugging Java applets

If an applet doesn't seem to be working properly, check if any error occurs while it's running. You can see what the applet writes to the standard output and error output by editing the batch files specified in the Application Path. It's highly recommended that you enable these features when curating Java applets.

startJava.bat

Open FPSoftware\startJava.bat and change "java" to "javadebug" in the following line:

FlashpointSecurePlayer -n "java" -sw %*

A separate window containing the applet's output will now show up everytime you run the AppletViewer.

startJavaInBrowser.bat

Open FPSoftware\startJava.bat and change OFF to ON in the following line:

SET FLASHPOINT_DEBUG_MODE=OFF

This will always show the Java console when the applet runs and will place a Java icon in the system tray, which you can use to bring up the console at any time.

Useful Tools and Resources

This section will link to external tools and resources that may be useful to learn more about the technology or to try to get certain games and animations working.

  • Bytecode Editors: dirtyJOE, JByteMod. Always check if there's a simpler way to get Java applets working. Only modify the original files as a last resort!

See also: Helpful Tools.

Troubleshooting

This section will cover some of the problems that may arise when curating Java games and animations. It's highly recommended that you enable the debugging options mentioned above when curating Java applets. If you're experiencing an issue not mentioned in this section be sure to ask for help in the #curator-lounge channel in the Flashpoint Discord server.

The applet closes immediately when I try to run it

The AppletViewer will close immediately if it can't find the specified file or if this file doesn't have an applet tag. You should check if the launch command is correct and if the file has a valid applet tag in it.

The applet is trying to request files that are suspiciously missing from the original site

If the class files are packaged in JAR or ZIP archives, be sure to open them and check if the missing files are there. If they are, extract them to the same directory as the archive. Some Java games have trouble loading assets from inside these archives and will instead try to load them externally. Note that you'll never have to extract class files.

The applet isn't displaying certain characters correctly

Try setting the correct character encoding and locale properties in the AppletViewer. For example, if a game from GeoCities Japan needed to display Japanese characters, you would add all of the following to the end of the game's launch command: -J-Dfile.encoding=UTF8 -J-Duser.language=ja -J-Duser.country=JP. If you're running the applet in a browser with startJavaInBrowser.bat, remove the -J prefix.

A comparision between using the English locale (left) and the Japanese locale (right)

You can also see what the current character encoding and locale settings are in the AppletViewer by using -J-XshowSettings. If you're running the applet in a browser with startJavaInBrowser.bat, don't use this option. Instead, enable the Java console and press 's' to show the current settings.

If you would like to know more about which character encodings and locales are supported in Java 8, see:

The applet ignores the Tab key's input

If the applet doesn't seem to be registering the Tab key's input, then follow the next steps:

  1. Download this ZIP which contains a wrapper applet that will fix this problem before loading the main applet.
  2. Put FlashpointDisableFocusTraversalKeysWrapperApplet.class next to the applet's class files or archives.
  3. Add the following parameter to the applet tag: <param name="flashpointClassToLoad" value="">, where the value is the one specified in the code attribute. Remove the trailing .class if there is one.
  4. Change the code attribute to FlashpointDisableFocusTraversalKeysWrapperApplet. The wrapper applet will run before loading the main one.

This wrapper disables all forward and backwards focus traversal keys (like Tab and Shift-Tab), so that these key events become available to the key event listeners. These keys would normally be used to change the keyboard focus in components like buttons and text fields. Applets affected by this problem expect the Tab key's input to work since they were made before Java 1.4 (2002) where this change was introduced.

Here's an example of a modified applet tag from the game Teslatron, where the second player uses the Tab key to drop bombs:

<applet code="FlashpointDisableFocusTraversalKeysWrapperApplet" width="523" height="367" archive="client_hacked.zip">
	<param name="flashpointClassToLoad" value="client.RobombClient">

	<param name="cabbase" value="client.cab">
	<param name="game" value="123">
	<param name="on-site" value="true">
	<param name="noPlayers" value="2">
</applet>

For more information on key listeners and focus traversal keys in Java, see the following pages:

The applet doesn't start in the AppletViewer and shows the following error: netscape.javascript.JSException

This means that the applet wants to interface with the browser, likely trying to execute some JavaScript code. This error may be ignored in some cases. For example, an applet that uses JavaScript to track the number of times the game has been played. If this doesn't impact the gameplay or show any side effects, you can safetly ignore it. Otherwise, you will need to run the applet in a browser. Refer to this section to learn how to do it.

The applet doesn't start and shows the following error: java.lang.ClassFormatError: Incompatible magic value 1008813135 in class file

This error means that the class file is in fact just an HTML file. This is likely due to being served a 404 page because the class file is missing. Try opening the file in a text editor or browser to confirm this. The value 1008813135 represents <!DO, in other words, the start of an HTML <!DOCTYPE> declaration. You may see a different magic value in the error depending on the wrong type of file you were served. The class file format is always supposed to have 3405691582 (0xCAFEBABE in hexadecimal) as the first four bytes.

The applet doesn't start and shows one of the following error: java.lang.ClassFormatError: Name index 0 in LocalVariableTable has bad constant type in class file

This error means that Java encountered a problem while checking if the class file is formatted correctly. You can fix this by adding the following option to the end of the launch command: -J-Xverify:none. If you're running the applet in a browser with startJavaInBrowser.bat, use -Xverify:none instead.

This command line argument tells Java to skip the bytecode verification step, meaning this error will be ignored and the applet can continue running. Note that this should be done sparingly, and only when you know it's an error of this nature. You should always check if the applet works correctly after using it.

This same fix also applies to the following errors: java.lang.ClassFormatError: Invalid start_pc 65535 in LocalVariableTable in class file and java.lang.ClassFormatError: Illegal field name "?" in class.

More information about this error: when compiling Java source files, you can specify certain debug options, one of which generates information about the local variables of each method (the LocalVariableTable attribute). This error happens when there's a single LocalVariableTable that is invalid according to the class file format. It seems to be most common with applets in the late 1990s and early 2000s which were compiled using older Java versions.

A comparision between a normal LocalVariableTable (left) and an invalid one (right) using dirtyJOE. Notice the use of an invalid descriptor (NULL) in this last one.

Developer Specific Troubleshooting

This section will cover some problems that commonly show up when curating Java games and animations from specific developers. This might also be useful if you're experiencing similar issues in applets made by developers that aren't listed here. If you're experiencing an issue not mentioned in this section be sure to ask for help in the #curator-lounge channel in the Flashpoint Discord server.

MumboJumbo

If a game crashes after your score reaches 1,000 points, try changing the locale property to English (US). You can do this by adding the following before the game's launch command: -J-Duser.language=en -J-Duser.country=US.

Note that you won't experience this issue if this is already your default locale. To avoid curating a game that might crash for another player, try playing the game with the following locale settings: -J-Duser.language=es -J-Duser.country=ES. You can see what the current locale settings are by using -J-XshowSettings.

Why this happens: the game uses NumberFormat.getInstance() to get a number format to format the player's score. This method uses the default locale, meaning that a number like one thousand is formatted as either 1,000 or 1.000 depending on your locale. The game's score font data is only defined for the numbers 0 through 9 and the comma, but not for the period. While the game is iterating over the formatted score string, the image data relative to the period will be returned as null, causing the game to eventually throw a NullPointerException when it accesses the image's attributes.

GameHouse

GameHouse Java games are both site and time locked, which means that they can only be played in certain domains and before a certain date. These game check where and when they're being played by requesting a file called license.txt which contains one or more license keys. Each key is associated with a domain name and expiration date. The idea was that game portals like Shockwave.com or Yahoo! Games would pay GameHouse to renew their licenses. If a game detects that it's being played after this date, then it won't start and will print the following:

Init 2.4
Date Value 1567034208000
Please Contact GameHouse, Inc. for configuration instructions.

If you're collecting a game's assets by using a tool like MAD4FP, you can set your system clock to the year 2000 before running it.

If you already have all the game's assets and want to curate it, change the contents of the license.txt to the following:

#gamehouse.com
Ajeaarvtzvfaapntzpjtziftzjgaakwszrgaabgaachaariszmesz

After doing this, you have to move the assets to gamehouse.com (and not www.gamehouse.com). This new license key has no date associated with it, and was meant for GameHouse to host their own games without having to worry about expiration dates. Since you are moving the files from their original source, you can use the following directory structure: gamehouse.com\Java\GameName.

If you'd like to learn more about GameHouse's Java games installation instructions, see this page.