wiki:PipeOnline/EclipseSetup
Last modified 9 years ago Last modified on 10/30/09 12:45:46

How to set up Eclipse with PipeOnline

Development Environment

The screenshots here are done using Eclipse 3.5 EE,

Java


Download and install Java SDK (Java SE Development Kit) if you haven't done so already. The latest versions should do fine, but compiler compliancy will have to be set to version 1.5 in Eclipse.

Eclipse

Download Eclipse IDE for Java EE Developers (http://www.eclipse.org/).
Extract to a folder of your choice.
Create a shortcut on the desktop and/or start menu for convenience. (A neater solution is to use the Eclipse Launcher which makes it easy to manage several Eclipse installations on the same computer.)
The first time you install Eclipse on the computer, you will be asked to choose a workspace (unless you're using the Eclipse Launcher ;)). Choose for instance C:\Users\<username>\workspaces\pipeonline . Remember to take note of these system-specific paths during the setup. They will not be the same on your system, and might even possibly vary throughout this guide. This workspace path is the one you'll most often have to refer to.

Subclipse

Begin the installation of Subclipse from the Eclipse Help menu item.
This screen is showing the New Remote Site dialog, filled in with the correct information to install Subclipse.

* Name: Subclipse 1.6.x (Eclipse 3.2+)
* Location: http://subclipse.tigris.org/update_1.6.x
Select the Subclipse entry and click Next.
This next screen shows all of the features that are available to install. Click Next.
Click the button to accept the license agreement and click Finish.
This may take some time to download.
There should be an Eclipse preference to turn off this next dialog. Signed features are very rare, not even Eclipse.org nor IBM sign their features.
Eclipse needs to be restarted after installing Subclipse.

Project Setup

Checkout

Projects can now be added by right clicking in the Package Explorer and choosing Import -> Checkout projects from SVN. PipeOnline is based on the Pipeline project, so the pipeline-core package will have to be imported from there. The pipeonline-core package builds on the pipeline-core package and implements core functionality like queues and other services to be run on the server. The web interface is implemented in pipeonline-web, which depends on both pipeline-core and pipeonline-core. pipeonline-core and pipeonline-web are imported from https://daisy-svn.cvsdude.com/pipeline and pipeline-core are imported from https://daisymfc.svn.sourceforge.net/svnroot/daisymfc . Import the projects individually!

pipeonline-core

Go to the workbench, right click in the package explorer and choose Import.
Choose Checkout projects from SVN
Choose to create a new repository location
Use the location https://daisy-svn.cvsdude.com/pipeline
Select the pipeonline-core folder and click Finish.

pipeonline-web

Right click in the package explorer and choose Import.
Choose Checkout projects from SVN.
Use the same repository as for pipeonline-core.
Select the pipeonline-web folder and click Finish.

pipeline-core

Right click in the package explorer and choose Import.
Choose Checkout projects from SVN.
Choose to create a new repository location.
Use the location https://daisymfc.svn.sourceforge.net/svnroot/daisymfc
Select the dmfc folder and click Next (if you accidentally click Finish, you can rename the project from org.daisy.pipeline to pipeline-core in the project explorer instead).
Set the Project Name to pipeline-core and click Finish.
This may take some time (it weighs about 175 MiB).

Dependencies

Extract the contents of http://home.samfundet.no/~josteija/javautils.zip javautils.zip to a folder which will contain all Java library dependencies. E.g. C:\javautils, C:\javalib, C:\Users\<username>\javautils or something similar. All the projects have dependencies to other java libraries, which will have to be placed here.
Open up the preferences.
In 'Classpath Variables', select New to add a new variable which you call LIB_HOME. The path should point to the directory of your java libraries, e.g. c:\javautils. Press OK and notice that the variable is added to the list of defined classpath variables. Click Ok to close the Preferences.

Each time you want to add a new library which is located underneath this path, choose to Extend this variable in the Add Variable dialogue under a projects build path (right click project -> Properties -> Java Build Path -> Libraries -> Add Variable -> LIB_HOME -> Extend). pipeonline-core/web uses this LIB_HOME variable to reference library dependencies in the project build path. If you don't have these libraries, locate them on the web and place them beneath your LIB_HOME folder as the build path of the respective project dictates.
You will have to do a full rebuild. Click Ok.

Project Properties

For all projects, set Project Facets first, then Java EE Module Dependencies.

Open project properties for pipeonline-web.
Go to Project Facets and select Dynamic Web Module, Java and JavaScript? Toolkit.
Check Project References and make sure that pipeline-core and pipeonline-core is selected (they should be so by default). Click Ok.
Again, open up project properties for pipeonline-web.
A Java EE Module Dependencies page will have appeared. Open it, click Select All (pipeline-core, pipeonline-core and ALL other jars available are now selected. Click Ok.
Open project properties for pipeonline-core.
Go to Project Facets and select Java, Java Persistence and Utility Module.
Check Project References and make sure that pipeline-core is selected while pipeonline-web is not (they should be so by default). Click Ok.
Again, open up project properties for pipeonline-core.
A Java EE Module Dependencies page will have appeared. Open it and click Select All, then click Ok.
Open project properties for pipeline-core.
Go to Project Facets and select Java and Utility Module (they might be so by default).
Check Project References and make sure that neither pipeonline-core nor pipeonline-web is selected (they should be so by default).
Go to Java EE Module Dependencies and select Plug-in Dependencies, onvdl.jar, saxon8.jar, stax-api-1.0.1.jar, wstx-lgpl-3.2.8.jar and xercesImpl.jar (these 5 last ones are contained in pipeline-core/lib).
Check that all previously referenced libraries are present underneath the LIB_HOME variable which you specified.

Personal settings for PipeOnline

Open settings.properties (pipeonline-core/src/META-INF) and set the following values:
- pipeline.dir (the path of pipeline-core)
- pipeonline.jobs.datadir (the path where PipeOnline jobs will be stored)

The values of this setup file will be inserted into the MySQL database table 'property' upon launching the server. If you forget to edit this file before running the server (or if you simply later want to change them), you will have to do so directly in the database. This is explained in the MySQL section below.

Compiler Compliance

Set the Java compiler compliance level to 1.5 in Eclipse.

Open Preferences.
Set compliance level to 1.5 on the Compiler page.
Do a full rebuild.

MySQL

Download and install MySQL Community Server from http://dev.mysql.com/downloads/. Make sure you assign an admin password during install.
Open the script populateDB.txt. For this script to work, you must first change the paths for the pipeline scripts to the ones which are valid for your system. Or, you can ignore these pipeline script inserts and add them later using the PipeOnline admin interface. You should add the 'user' and 'admin' users, so that you can log on to PipeOnline. Passwords are also 'user' and 'admin' respectively.
Open the MySQL Command Line Client.
Enter your password (if you didn't set one, it is most likely 'root'), and run the following commands (insert your workspace-path):

source <WORKSPACE>\pipeonline-core\database\createDB.txt
source <WORKSPACE>\pipeonline-core\database\createUser.txt
source <WORKSPACE>\pipeonline-core\database\createTables.txt
source <WORKSPACE>\pipeonline-core\database\populateDB.txt

Apache Tomcat

Installation

Download Apache Tomcat (http://tomcat.apache.org/download-60.cgi, select Binary Core Zip).
Extract to a folder of your choice.
Enable UTF-8 support in Tomcat to handle Norwegian characters. Open server.xml in the Apache configuration directory, e.g. C:\apache-tomcat-6.0.x\conf\server.xml. Locate all <Connector> elements and append URIEncoding="UTF-8" to each <Connector> element. An example:

<Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" URIEncoding="UTF-8"/>

This procedure is described in more detail here: http://wiki.apache.org/tomcat/FAQ/CharacterEncoding

Server in Eclipse

Open the Servers view (Window -> Show View -> Server -> Servers).
The Server tab is now visible. Right click in the Server tab and select New -> Server.
Select Apache Tomcat v.6.0. Let the server's host name be 'localhost' (default setting).
Browse to the Apache Tomcat install directory.
In the last step, select pipeonline-web and move it over to the configured window. A 'Servers' should appear in the Package Explorer..
Now, just start the server once. Right click on pipeonline-web, and under Run as - select Run on Server.
By default, the Tomcat server we just defined should be selected. In addition, tick "Always use this server when running this project", then click Finish.
Publishing may take a minute or so...
And since you're starting a web server, the Windows Firewall (or another firewall if you've got one installed) may ask for your permission to allow it to run.
You will get a 404 Page Not Found this first time around. But having run the server once will generate a Run Configuration for this particular server which we will have to edit. Select the Console tab and stop the server. Then go to Run Configurations (Run -> Run Configurations).
There should now be a configuration for the Apache Tomcat you just installed. If you kept default settings, this should be called 'Tomcat v6.0 Server at localhost'. Go to the Arguments tab. In VM arguments, there are already some arguments present, but we need to append the following: -javaagent:C:\<LIB_HOME>\spring-agent.jar -Xmx512m

This path depends on which folder you have set to LIB_HOME, in this example it is C:\javautils. Copy this line (including the hyphen (-) in the beginning) and paste it into the VM Arguments window after the existing arguments on a new line. The Xmx512m command sets maximum heap size to 512 MB for the web server. PipeOnline cannot run on the default heap size (128MB).

The VM Arguments window should now contain something like this:

-Dcatalina.base="C:\Users\jostein.NLB\workspaces\pipeonline2\.metadata\.plugins\org.eclipse.wst.server.core\tmp0" -Dcatalina.home="C:\apache-tomcat-6.0.20" -Dwtp.deploy="C:\Users\jostein.NLB\workspaces\pipeonline2\.metadata\.plugins\org.eclipse.wst.server.core\tmp0\wtpwebapps" -Djava.endorsed.dirs="C:\apache-tomcat-6.0.20\endorsed" -javaagent:C:\Users\jostein.NLB\javautils\spring-agent.jar -Xmx512m

Click Apply and then Close (don't Run it yet, we will have to make some changes first).

Increasing Server Timeout

PipeOnline sometimes needs more than 45 seconds (default) to start up. Stop the server and go to he Servers tab.
Double click the server in the Servers tab. An overview of the server settings is presented. There is an expandible tab here called 'Timeouts'. Set the start value to e.g. 300 or any other amount which gives the server a sufficient temporal buffer.

Save the settings and close Eclipse.

Setting the Deployment Descriptor

The Deployment Descriptor (DD) contains information about how to deploy this application. PipeOnline-web has a DD which is located at pipeonline-web/web/WEB-INF/web.xml.

Most likely, your Eclipse setup will have made a default DD for you, located in pipeonline-web/WebContent.

If you wish, you can check this in the Java EE Perspective (Window -> Open Perspective). Expand pipeonline-web and single click the Deployment Descriptor. Check its value which is displayed at the bottom left of your Eclipse window.

To change its value, go to the file 'org.eclipse.wst.common.component' which is located in the .settings folder of your pipeonline-web project (<WORKSPACE>/pipeonline-web/.settings/). Make sure that Eclipse is closed first. Manually edit this file, changing the value of this line:

<wb-resource deploy-path="/" source-path="/web"/>

so that the source-path is set to "/web".

Eclipse presupposes that the DD is located at "/WEB-INF/web.xml" underneath this source path you just set.

Save the file, start Eclipse and if you want; check that the recent change is reflected in pipeonline-web's DD.

Running PipeOnline

Right click on the server in the Server tab and choose 'Start'. Pay attention to the console window and keep your fingers crossed. Be on the lookout for error messages and stack traces. If there is no sign of error, congratulations!

While the server is running, you can also access PipeOnline from your favourite web browser by going to the address http://localhost:8080/pipeonline-web/

Other

Updating absolute paths in database

Open MySQL Command Line Client. Write:

use pipeonline;
select * from property;

Check that the values for pipeline.dir and pipeonline.jobs.datadir are set to folders which exist on your computer. To change these paths, write something like this:

update property set value='C:/workspace/pipeline-core' where name='pipeline.dir';
update property set value='C:/Pipeline-jobs' where name='pipeline.jobs.datadir';

If clicking on a script results in a "The system cannot find the path specified" error (in the stacktrace), then the (absolute) path to the script in the database is probably also wrong. The three default scripts can be updated like this:

update scriptfile set uri='file:///workspace/pipeline-core/scripts/verify/DTBookValidator.taskScript' where name='DTBook Validator';
update scriptfile set uri='file:///workspaces/pipeline-core/scripts/create_distribute/dtbook/RtfToDtbook.taskScript' where name='RTF to DTBook';
update scriptfile set uri='file:///workspaces/pipeline-core/scripts/create_distribute/dtb/Narrator-DtbookToDaisy.taskScript' where name='TTS Narrator';

Disabling validation

Validation is enabled for the projects, which will take some time to run and may generate errors. To suspend all validators enter the project preferences to pipeline-core pipeonline-core and pipeonline-web. Select Validation, Enable project specific settings, Suspend all validators and click Ok.

Speech synthesis

Start by installing/extracting the LAME MP3 encoder (download from for instance http://www.free-codecs.com/download/Lame_Encoder.htm and extract to a folder of your choice).

Open pipeline.user.properties (pipeline-core/src) and set the path to the LAME MP3 encoder.

Attachments