Java Curation: Difference between revisions

From Flashpoint Datahub
Jump to navigation Jump to search
mNo edit summary
mNo edit summary
Line 292: Line 292:
http://www.shockwave.com/content/cubis/sis/images/rim_b.jpg
http://www.shockwave.com/content/cubis/sis/images/rim_b.jpg
</pre>
</pre>
To download these assets for your Java curation, create a text file with these URLs and then add <code>http://web.archive.org/web/1id_/</code> to the beginning of each one. The <code>1id_</code> segment tells the Wayback Machine to use the most recent capture of the file without any modifications. Then, drag the text file onto the [[Helpful Tools#Software for Archiving|cURLsDownloader]] tool to start downloading them (read its help file for more details). For more information on the Wayback CDX Server API, refer to the [https://github.com/internetarchive/wayback/tree/master/wayback-cdx-server documentation on its GitHub].
To download these assets for your Java curation, create a text file with these URLs and then add <code>http://web.archive.org/web/1id_/</code> to the beginning of each one. The <code>1id_</code> segment tells the Wayback Machine to use the original file, without any modifications. Then, drag the text file onto the [[Helpful Tools#Software for Archiving|cURLsDownloader]] tool to start downloading them (read its help file for more details). For more information on the Wayback CDX Server API, refer to the [https://github.com/internetarchive/wayback/tree/master/wayback-cdx-server documentation on its GitHub].


== Useful tools and resources ==
== Useful tools and resources ==

Revision as of 21:14, 23 May 2020

This guide will show you how to curate Java games and animations using Flashpoint Core, as well as dealing with any problems that may come up in the process.

This page is accurate as of Flashpoint Core 5.

Recognizing Java applets

Unlike other technologies such as Flash or Shockwave, Java applets are most commonly embedded on a page using an applet tag rather than an object/embed tag. Finding these tags is important since they're necessary to get the games running in Flashpoint.

Here's an example of the most basic type you can find:

<applet code="iceblox.class" width="390" height="350"></applet>

Here's a more complex example that shows certain parameters being passed to the game and that the class files may be packaged in a JAR/ZIP/CAB archive. If you tried to run this game on a browser that didn't support Java, you'd only see "Unfortunately, your browser does not support Java". This can be a good way to know if the web game or animation you're trying to curate uses Java.

<applet code="gamehouse.SuperApplet.class" archive="Collapse.zip" height="400" width="460" viewastext mayscript>
	<param name="cabbase" value="Collapse.cab">
	<param name="canvas" value="Collapse.CollapseCanvas">
	<param name="bgcolor" value="FFFFFF">
	<param name="textcolor" value="000000">
	<param name="buyurl" value="http://get.games.yahoo.com/proddesc?gamekey=spcollapse">
	<param name="target" value="_blank">

	Unfortunately, your browser does not support Java.
</applet>

Java applets may also be embedded using the object and embed tags, although this appears to be less common:

<object classid="clsid:8AD9C840-044E-11D1-B3E9-00805F499D93" codebase="http://java.sun.com/products/plugin/1.2/jinstall-12-win32.cab#Version=1,2,0,0" width="100%" height="100%" name="Rockz" alt="Java applet failure">
	<param name="code" value="rockz.Run.class" >
	<param name="archive" value="rockz.jar" >
	<param name="name" value="Rockz" >
	<param name="type" value="application/x-java-applet;version=1.2">

	<embed code="rockz.Run.class" archive="rockz.jar" name="Rockz" type="application/x-java-applet;version=1.2" width="100%" height="100%" alt="Java applet failure" pluginspage="http://java.sun.com/products/plugin/1.2/plugin-install.html"></embed>
</object>

To find these tags, right-click the page with the applet, click "Inspect Element" or "View Page Source" and then search for "applet".

Curation steps

Flashpoint runs Java applets offline by using an application called the AppletViewer. You can think of it like the equivalent to a Flash projector. Let's take a look at a simple example by curating the game "Realspace 2 - Emperor's Revenge" from https://www.miniclip.com/games/real-space-2/en/

  1. Open this page in your browser. You should see the text saying that Java is not enabled or installed. This indicates that it's a Java game, so we should take a look at the page's source and try to find the applet tag. If we search for "applet", we'll find the following:
    <applet width="500" height="500" code="gameapp.class" archive="gameapp.jar">
    	<param name="SIZE_SCALE" value="1.0" />
    	If you see this message you do not have Java enabled or do not have Java installed.</br><a href="http://wwww.java.com">Please see Java.com</a> to download the latest version of Java.</a>
    </applet>
    

    Note that the page could have also written the tag dynamically which means that you wouldn't be able to find it by just checking its source. In this case you would use your browser's "Inspect Element" option and search for it there.

  2. In your Flashpoint Core directory, navigate to Server\htdocs. In htdocs, create a series of directories that match the original site's structure. In this case, we'd create four directories, each inside the previous one: www.miniclip.com\games\real-space-2\en\
  3. Navigate to the last directory (in this case \en\) and create the file that will contain the applet tag we found in step 1. You should give it the same name as the original site. In this case there isn't one, so we'll just create a file called index.html.
  4. Open this file you've just created in a text editor and paste the applet tag in there.
    <applet width="500" height="500" code="gameapp.class" archive="gameapp.jar">
    	<param name="SIZE_SCALE" value="1.0" />
    </applet>
    
  5. Download the game's assets (code, images, sounds, etc) and put them in the correct directories so that they also match the original site's structure. It's possible that all the files needed are inside the archive (called gameapp.jar here). If they were and we didn't need any additional files then we could move onto the next step. However, for this and most other Java games, the archive only has the game's code. We still need to download the rest of the assets. This can be done in several ways, including:
    • Checking what the game is requesting by running it, looking at the Logs tab in the Flashpoint launcher, and downloading these files manually.
    • Using a tool like wget or cURLsDownloader to download the assets from specific URLs. This is especially useful if you want to download the files from the Wayback Machine (see this section for more details).
    • Using a tool that automatically downloads these files from the original site as they're requested by the game. See the MAD4FP page for more details on how to do this.
    Some of the files requested by "Realspace 2 - Emperor's Revenge" in the Logs tab of the Flashpoint launcher.
  6. You can now test your curation in Flashpoint Core. Go to the Curate tab and press the New Curation button on the top right corner. Enter the game's title, platform, application path, and launch command in the respective text fields. In this case, they would be:
    Title: Realspace 2 - Emperor's Revenge
    Platform: Java
    Application Path: FPSoftware\startJava.bat
    Launch Command: http://www.miniclip.com/games/real-space-2/en/index.html
    

    The platform should be the same for all Java curations. Make sure that the launch command starts with http:// and not https://

    You can start the game by pressing the Run button at the bottom of your new curation. You should now see the game running in the AppletViewer:

    "Realspace 2 - Emperor's Revenge" running in the AppletViewer.
  7. Fill the rest of the fields and submit your curation by following the Curation Format. For more information on how to curate games for Flashpoint, refer to the Curation Tutorial.

Curation tips

  • You may sometimes see a parameter in the applet tag called cabbase that specifies a CAB file. This was used to allow the same applet to run in different browsers - Netscape Navigator would download the JAR or ZIP in the archive attribute while Internet Explorer would download the CAB in the cabbase parameter. Although the AppletViewer doesn't request this file, you should still include it in your submission for preservation's sake.
  • The AppletViewer ignores everything in the file it reads except for the tags that embed a Java applet. You can see the tag that was read by the AppletViewer by going to Applet > Tag...
  • You can sometimes find more information about the author, version and copyright of an applet by going to Applet > Info... in the AppletViewer.
  • If a Java game requests external assets during gameplay and you don't have time to play through the entire game, try opening the game's .class or .jar files with a Java decompiler such as JD-GUI. Images and sounds are often loaded using Applet.getImage() and Applet.getAudioClip(), respectively. Alternatively, you can open them in a hex editor such as HxD, though this will make it hard to know how and when the game is requesting assets. References to external assets are often plain text strings, so you may try searching for common asset types such as .gif and .jpg images or .au sounds.
    Regardless of the method used, you can make a list of these assets, then download them from the internet and add them to the game's directory. All of these tools are linked in this section.

Application paths

This section will list all the possible values for the Application Path field in the metadata and launcher. Both of these currently use Java 8 update 181. If you encounter any problems that you think might be solved by using an older Java version, read the steps in the troubleshooting section first. Note that Flashpoint doesn't currently support Java Web Start or Microsoft Java.

startJava.bat

Use FPSoftware\startJava.bat to run the applet in the AppletViewer (as we saw in the example above). If you use this application path, put the URL to the file that contains the <applet> tag in the Launch Command field. For example:

Application Path: FPSoftware\startJava.bat
Launch Command: http://www.example.com/game/index.html

If the URL contains spaces, replace them with %20. For example:

Launch Command: http://www.example.com/game/index%20with%20spaces.html

You can also pass extra command line arguments to the Java Virtual Machine (JVM) by placing them after the URL. These options must be prefixed with -J. For example:

Launch Command: http://www.example.com/game/index.html -J-Duser.language=en -J-Duser.country=US

Read the troubleshooting section to see how these may be used to solve some common problems.

If the URL contains an ampersand character (&), wrap it in double quotes. This happens when you use PHP to set certain parameters dynamically in the applet tag. For example:

Launch Command: "http://www.example.com/game/index.php?param1=value1&param2=value2"

Where "value1" and "value2" could change what the parameters are set to.

startJavaInBrowser.bat

Use FPSoftware\startJavaInBrowser.bat to run the applet in a browser. Unlike the application path mentioned above, you don't have to necessarily put the URL to the file that contains the <applet> tag in the Launch Command field. For example:

Application Path: FPSoftware\startJavaInBrowser.bat
Launch Command: http://www.example.com/game/index.html

The previous notes about the launch commands also apply, with the exception of the extra command line arguments. If you use startJavaInBrowser.bat, these options are not prefixed with -J. For example:

Launch Command: http://www.example.com/game/index.html -Duser.language=en -Duser.country=US

In addition to this, you must always follow the next two steps. These will ensure that the applet is allowed to run in its domain with the correct Java version.

  1. Add the game's domain to the end of the exception site list file in FPSoftware\Java\JDK_1.8.0_181\jre\lib\exception.sites. For this example, this would be http://www.example.com/. If you're curating multiple games from the same domain, you only have to do this once. Be sure to include this information in the Curation Notes field in the metadata. For example:
    Before testing the game, add http://www.example.com/ to the exception site list file in FPSoftware\Java\JDK_1.8.0_181\jre\lib\exception.sites
    
  2. Add these two parameters to the <applet> tag:
    <param name="java_version" value="1.8.0_181">
    <param name="separate_jvm" value="true">
    

    If the applet uses an <embed> tag instead, these parameters would look like:

    <embed java_version="1.8.0_181" separate_jvm="true">
    

    These parameters have no effect if the same applet is launched in the AppletViewer via startJava.bat.

Every applet that runs in a browser will show a security prompt that must be accepted before starting. To make this clear to Flashpoint users, add the following to the beginning of the Game Notes field in the metadata:

This Java game will run in a browser. Make sure to accept any security prompts that show up while playing.

Since in-browser Java applets are currently only supported on Windows, you should provide an option to launch them in the AppletViewer (if possible). You can do so by making an additional application called Play in the AppletViewer with the startJava.bat application path, and by adding the following to the Game Notes field in the metadata:

If playing in a browser doesn't work, try using the "Play in the AppletViewer" option under Additional Applications.

When to run applets in a browser

You should run Java applets in a browser if any of the following conditions are met:

  • The page has two or more applets that communicate via JavaScript. Note that it's possible for multiple applets to communicate with each other using the AppletViewer, provided they do so using the AppletContext.getApplet() method.
  • The applet relies on page element interaction (e.g. pressing buttons) to be played.
  • The applet uses browser cookies to save the game's progress. These are saved and loaded using JavaScript.
  • The page contains other multimedia elements that enhance the experience. For example, the page might embed Flash or Shockwave movies, or play music (MIDI, MOD).

These aren't hard requirements, you can curate any applet using startJavaInBrowser.bat if you feel like it captures the original experience. If you decide to curate an applet using this application path, you should also include any other assets (images, sounds) that are embedded on the same page.

Browser curation tips

If some JavaScript code on the page is meant to interact with the applet, you should check how it's doing so. Older applets may rely on old JavaScript functionalities that modern browsers don't support. For example, in the game Urbanoids, the player has to click a button on the page to start the game:

function start_it() {
	document.game.clickStart();
}

<applet code="noids.class" width=240 height=263 name="game">
	<param name="java_version" value="1.8.0_181">
	<param name="separate_jvm" value="true">
</applet>

This wouldn't work because the applet tag uses the attribute name. To fix this, we would need to change it to id:

<applet code="noids.class" width=240 height=263 id="game">

Debugging Java applets

If an applet doesn't seem to be working properly, check if any error occurs while it's running. You can see what the applet writes to the standard output and error output by editing the batch files specified in the Application Path. It's highly recommended that you enable these features when curating Java applets.

startJava.bat

Open FPSoftware\startJava.bat and change "java" to "javadebug" in the following line:

FlashpointSecurePlayer -n "java" -sw %*

A separate window containing the applet's output will now show up everytime you run the AppletViewer.

startJavaInBrowser.bat

Open FPSoftware\startJavaInBrowser.bat and change OFF to ON in the following line:

SET FLASHPOINT_DEBUG_MODE=OFF

This will always show the Java console when the applet runs and will place a Java icon in the system tray, which you can use to bring up the console at any time.

Finding Java applets in the Wayback Machine

Since Java applets were first introduced in 1995, it can sometimes be hard to locate files belonging to older games. This section will show you how to more easily find these by using the Wayback CDX Server API. For example, if you wanted to see what Java games hosted on Shockwave.com were saved by the Wayback Machine, you could use the following query:

https://web.archive.org/cdx/search/cdx?url=*.shockwave.com/&filter=original:.*(\.class|\.jar|\.zip|\.cab|\.au).*&fl=original,timestamp,statuscode&collapse=urlkey

Where:

  • url=*.shockwave.com/ says to search in every Shockwave.com subdomain.
  • filter=original:.*(\.class|\.jar|\.zip|\.cab|\.au).* says to only show the URLs that contain these file extensions: .class, .jar, .zip, .cab, and .au.
  • fl=original,timestamp,statuscode says to show three fields in this order: the URL, timestamp, and HTTP status code.
  • collapse=urlkey says to only show unique URLs.

Now that we have this list, we can scan it for Java games. For example, the game Cubis:

http://www.shockwave.com/content/cubis/sis/cubis.jar 20180828052922 200
http://www.shockwave.com/content/cubis/sis/sounds/allcubes.au 20180828054857 200
http://www.shockwave.com/content/cubis/sis/sounds/blank.au 20180828055355 200

Note that this game also uses .jpg and .gif images. We didn't include these file extensions in the query above since it would also include every unrelated image from Shockwave.com. Now that we know where to look, we can locate every file in the game's directory with the following query:

https://web.archive.org/cdx/search/cdx?url=www.shockwave.com/content/cubis/sis/*&fl=original&collapse=urlkey

Where:

  • url=www.shockwave.com/content/cubis/sis/* says to show every URL with that prefix.

We can now see the additional files that were excluded by the first query:

http://www.shockwave.com/content/cubis/sis/images/o_solid.gif
http://www.shockwave.com/content/cubis/sis/images/panel.jpg
http://www.shockwave.com/content/cubis/sis/images/piece_marker.gif
http://www.shockwave.com/content/cubis/sis/images/rim_b.jpg

To download these assets for your Java curation, create a text file with these URLs and then add http://web.archive.org/web/1id_/ to the beginning of each one. The 1id_ segment tells the Wayback Machine to use the original file, without any modifications. Then, drag the text file onto the cURLsDownloader tool to start downloading them (read its help file for more details). For more information on the Wayback CDX Server API, refer to the documentation on its GitHub.

Useful tools and resources

This section will link to external tools and resources that may be useful to learn more about the technology or to try to get certain games and animations working.

  • Bytecode Editors: dirtyJOE, JByteMod. Always check if there's a simpler way to get Java applets working. Only modify the original files as a last resort!

See also: Helpful Tools.

Troubleshooting

This section will cover some of the problems that may arise when curating Java games and animations. It's highly recommended that you enable the debugging options mentioned above when curating Java applets. If you're experiencing an issue not mentioned in this section be sure to ask for help in the #curator-lounge channel in the Flashpoint Discord server.

The applet closes immediately when I try to run it

The AppletViewer will close immediately if it can't find the specified file or if this file doesn't have an applet tag. You should check if the launch command is correct and if the file has a valid applet tag in it.

The applet is trying to request files that are suspiciously missing from the original site

If the class files are packaged in JAR or ZIP archives, be sure to open them and check if the missing files are there. If they are, extract them to the same directory as the archive. Some Java games have trouble loading assets from inside these archives and will instead try to load them externally. Note that you'll never have to extract class files.

The applet isn't displaying certain characters correctly

Try setting the correct character encoding and locale properties in the AppletViewer. For example, if a game from GeoCities Japan needed to display Japanese characters, you would add all of the following to the end of the game's launch command: -J-Dfile.encoding=UTF8 -J-Duser.language=ja -J-Duser.country=JP. If you're running the applet in a browser with startJavaInBrowser.bat, remove the -J prefix.

A comparision between using the English locale (left) and the Japanese locale (right)

You can also see what the current character encoding and locale settings are in the AppletViewer by using -J-XshowSettings. If you're running the applet in a browser with startJavaInBrowser.bat, don't use this option. Instead, enable the Java console and press 's' to show the current settings.

If you would like to know more about which character encodings and locales are supported in Java 8, see:

The applet ignores the Tab key's input

If the applet doesn't seem to be registering the Tab key's input, then follow the next steps:

  1. Download this ZIP which contains a wrapper applet that will fix this problem before loading the main applet.
  2. Put FlashpointDisableFocusTraversalKeysWrapperApplet.class next to the applet's class files or archives.
  3. Add the following parameter to the applet tag: <param name="flashpointClassToLoad" value="">, where the value is the one specified in the code attribute. Remove the trailing .class if there is one.
  4. Change the code attribute to FlashpointDisableFocusTraversalKeysWrapperApplet. The wrapper applet will run before loading the main one.

This wrapper disables all forward and backwards focus traversal keys (like Tab and Shift-Tab), so that these key events become available to the key event listeners. These keys would normally be used to change the keyboard focus in components like buttons and text fields. Applets affected by this problem expect the Tab key's input to work since they were made before Java 1.4 (2002) where this change was introduced.

Here's an example of a modified applet tag from the game Teslatron, where the second player uses the Tab key to drop bombs:

<applet code="FlashpointDisableFocusTraversalKeysWrapperApplet" width="523" height="367" archive="client_hacked.zip">
	<param name="flashpointClassToLoad" value="client.RobombClient">

	<param name="cabbase" value="client.cab">
	<param name="game" value="123">
	<param name="on-site" value="true">
	<param name="noPlayers" value="2">
</applet>

For more information on key listeners and focus traversal keys in Java, see the following pages:

The applet doesn't work properly in the AppletViewer and shows the following error: netscape.javascript.JSException

This means that the applet wants to interface with the browser, likely trying to execute some JavaScript code. This error may be ignored in some cases. For example, an applet that uses JavaScript to track the number of times the game has been played. If this doesn't impact the gameplay or show any side effects, you can safetly ignore it. Otherwise, you will need to run the applet in a browser. Refer to this section to learn how to do it.

The applet doesn't start and shows the following error: java.lang.ClassFormatError: Incompatible magic value 1008813135 in class file

This error means that the class file is in fact just an HTML file. This is likely due to being served a 404 page because the class file is missing. Try opening the file in a text editor or browser to confirm this. The value 1008813135 represents <!DO, in other words, the start of an HTML <!DOCTYPE> declaration. You may see a different magic value in the error depending on the wrong type of file you were served. The class file format is always supposed to have 3405691582 (0xCAFEBABE in hexadecimal) as the first four bytes.

The applet doesn't start and shows the following error: java.lang.ClassFormatError: Name index 0 in LocalVariableTable has bad constant type in class file

This error means that Java encountered a problem while checking if the class file is formatted correctly. You can fix this by adding the following option to the end of the launch command: -J-Xverify:none. If you're running the applet in a browser with startJavaInBrowser.bat, use -Xverify:none instead.

This command line argument tells Java to skip the bytecode verification step, meaning this error will be ignored and the applet can continue running. Note that this should be done sparingly, and only when you know it's an error of this nature. You should always check if the applet works correctly after using it.

This same fix also applies to the following errors:

java.lang.ClassFormatError: Invalid start_pc 65535 in LocalVariableTable in class file
java.lang.ClassFormatError: Illegal field name "?" in class

More information about this error: when compiling Java source files, you can specify certain debug options, one of which generates information about the local variables of each method (the LocalVariableTable attribute). This error happens when there's a single LocalVariableTable that is invalid according to the class file format. It seems to be most common with applets in the late 1990s and early 2000s which were compiled using older Java versions.

A comparision between a normal LocalVariableTable (left) and an invalid one (right) using dirtyJOE. Notice the use of an invalid descriptor (NULL) in this last one.

Developer specific troubleshooting

This section will cover some problems that commonly show up when curating Java games and animations from specific developers. This might also be useful if you're experiencing similar issues in applets made by developers that aren't listed here. If you're experiencing an issue not mentioned in this section be sure to ask for help in the #curator-lounge channel in the Flashpoint Discord server.

MumboJumbo

If a game crashes after your score reaches 1,000 points, try changing the locale property to English (US). You can do this by adding the following after the game's launch command: -J-Duser.language=en -J-Duser.country=US. If you're running the applet in a browser with startJavaInBrowser.bat, remove the -J prefix.

Note that you won't experience this issue if this is already your default locale. To avoid curating a game that might crash for another player, try playing the game with the following locale settings: -J-Duser.language=es -J-Duser.country=ES. You can also see what the current character encoding and locale settings are in the AppletViewer by using -J-XshowSettings. If you're running the game in a browser with startJavaInBrowser.bat, don't use this option. Instead, enable the Java console and press 's' to show the current settings.

Why this happens: the game uses NumberFormat.getInstance() to get a number format to format the player's score. This method uses the default locale, meaning that a number like one thousand is formatted as either 1,000 or 1.000 depending on your locale. The game's score font data is only defined for the numbers 0 through 9 and the comma, but not for the period. While the game is iterating over the formatted score string, the image data relative to the period will be returned as null, causing the game to eventually throw a NullPointerException when it accesses the image's attributes.

GameHouse

GameHouse Java games are both site and time locked, which means that they can only be played in certain domains and before a certain date. These game check where and when they're being played by requesting a file called license.txt which contains one or more license keys. Each key is associated with a domain name and expiration date. The idea was that game portals like Shockwave.com or Yahoo! Games would pay GameHouse to renew their licenses. If a game detects that it's being played after this date, then it won't start and will print the following:

Init 2.4
Date Value 1567034208000
Please Contact GameHouse, Inc. for configuration instructions.

If you're collecting a game's assets by using a tool like MAD4FP, you can set your system clock to the year 2000 before running it.

If you already have all the game's assets and want to curate it, change the contents of the license.txt to the following:

#gamehouse.com
Ajeaarvtzvfaapntzpjtziftzjgaakwszrgaabgaachaariszmesz

After doing this, you have to move the assets to gamehouse.com (and not www.gamehouse.com). This new license key has no date associated with it, and was meant for GameHouse to host their own games without having to worry about expiration dates. Since you are moving the files from their original source, you can use the following directory structure: gamehouse.com\Java\GameName.

If you'd like to learn more about GameHouse's Java games installation instructions, see this page.