Now we will see how to go further and develop a database Application with the same dual approach. We will use EMF and Teneo for this, in the same way showed in the series of tutorials “From Model to Persistence” at page EMF Development.

In the following instructions we must make a clear distinction between installing some feature into the IDE or into the Target Platform (RAP in our case). Installing into IDE is performed going to menu Help -> Install New Software…, while installing on the Target Platform is made with Window -> Preferences -> Plug-in Development -> Target Platform (see article Single sourcing with Eclipse RAP).

Let’s first start installing on our IDE all needed tools:

Update site: Indigo (included) – http://download.eclipse.org/releases/indigo
Category: Modeling
Feature: EMF – Eclipse Modeling Framework SDK

Update site: Indigo (included) – http://download.eclipse.org/releases/indigo
Category: General Purpose Tools
Feature: All SWT Designer features except SWT Designer XWT Support (requires Eclipse WTP/WST) and all WindowBuilder features except WindowBuilder XML Core

Update site: Elver (to be added) – http://www.elver.org/eclipse/update
Category: Teneo/Texo Dependencies
Feature: All

Update site: Teneo 1.2.0 (to be added) – http://download.eclipse.org/modeling/emf/teneo/updates/1.2.0/interim/
Category: EMF Teneo – Hibernate
Feature: All

Now let’s add on the RAP Target Platform the following (please consider that we already installed Rich Ajax Platform (RAP) Target Components and EMF RAP Target Components in previous tutorial):

Software site: Elver (to be added) – http://www.elver.org/eclipse/update
Category: Teneo/Texo Dependencies
Feature: All

Software site: Teneo 1.2.0 (to be added) – http://download.eclipse.org/modeling/emf/teneo/updates/1.2.0/interim/
Category: EMF Teneo – Hibernate
Feature: All

Ok now we are ready for the single sourcing development phase. For the model we can just Import the model plug-in we made in the tutorial Eclipse EMF: updating the Model. While we can modify our current RCP/RAP application (it.rcpvision.rcprap.application) in the following way.

In the Activator let’s connect to the database and load data (there is a populating section just for the very first time, when the database is not yet created)

import java.io.IOException;
import java.util.Properties;

import library.Author;
import library.Library;
import library.LibraryFactory;
import library.LibraryPackage;

import org.eclipse.emf.common.util.URI;
import org.eclipse.emf.ecore.EPackage;
import org.eclipse.emf.ecore.resource.Resource;
import org.eclipse.emf.ecore.resource.ResourceSet;
import org.eclipse.emf.ecore.resource.impl.ResourceSetImpl;
import org.eclipse.emf.teneo.hibernate.HbDataStore;
import org.eclipse.emf.teneo.hibernate.HbHelper;
import org.eclipse.emf.teneo.hibernate.resource.HibernateResource;
import org.eclipse.jface.resource.ImageDescriptor;
import org.eclipse.ui.plugin.AbstractUIPlugin;
import org.hibernate.cfg.Environment;
import org.osgi.framework.BundleContext;

...

	public void start(BundleContext context) throws Exception {
		super.start(context);
		plugin = this;

		loadResource();
	}

	private void loadResource() {
		// *************** Initialize Teneo Hibernate DataStore *************************************
		HbDataStore hbds = (HbDataStore) HbHelper.INSTANCE.createRegisterDataStore("MyDb");
		// Set Database properties
		Properties props = new Properties();
		props.setProperty(Environment.DRIVER, "com.mysql.jdbc.Driver");
		props.setProperty(Environment.URL, "jdbc:mysql://localhost:3306/test5?createDatabaseIfNotExist=true");
		props.setProperty(Environment.USER, "root");
		props.setProperty(Environment.PASS, "admin");
		props.setProperty(Environment.DIALECT, org.hibernate.dialect.MySQL5Dialect.class.getName());
		props.setProperty(Environment.SHOW_SQL, "true");
		props.setProperty(Environment.HBM2DDL_AUTO, "update");
		hbds.setDataStoreProperties(props);
		// Register EMF package
		hbds.setEPackages(new EPackage[] { LibraryPackage.eINSTANCE });
		hbds.initialize();

		// *************** Initialize Database Content Data *************************************
		// (the first time a new Library object container is persisted, otherwise it is just loaded)
		String uriStr = "hibernate://?"+HibernateResource.DS_NAME_PARAM+"=MyDb";
		final URI uri = URI.createURI(uriStr);
		ResourceSet resourceSet = new ResourceSetImpl();
		resource = resourceSet.createResource(uri);
		try {
		    resource.load(null);
		    if (resource.getContents().size() == 0) {
		    	Library library = LibraryFactory.eINSTANCE.createLibrary();
		        resource.getContents().add(library);

	            Author author = LibraryFactory.eINSTANCE.createAuthor();
	            author.setName("Ed");
	            author.setSurname("Merks");
	            library.getListAuthor().add(author);

	            author = LibraryFactory.eINSTANCE.createAuthor();
	            author.setName("Lars");
	            author.setSurname("Vogel");
	            library.getListAuthor().add(author);

	            author = LibraryFactory.eINSTANCE.createAuthor();
	            author.setName("Eike");
	            author.setSurname("Stepper");
	            library.getListAuthor().add(author);

	            author = LibraryFactory.eINSTANCE.createAuthor();
	            author.setName("Ralf");
	            author.setSurname("Sternberg");
	            library.getListAuthor().add(author);

	            author = LibraryFactory.eINSTANCE.createAuthor();
	            author.setName("Tom");
	            author.setSurname("Schindl");
	            library.getListAuthor().add(author);

	            resource.save(null);
		    }
		} catch (IOException e) {
		    // TODO Auto-generated catch block
		    e.printStackTrace();
		}
	}

now rewrite the View like this (as you have learnt in other our tutorials, here we put a TableViewer and bind the content of the loaded EMF resource)

package it.rcpvision.rcprap.application;

import library.Author;
import library.Library;

import org.eclipse.core.databinding.DataBindingContext;
import org.eclipse.core.databinding.beans.PojoObservables;
import org.eclipse.core.databinding.observable.list.WritableList;
import org.eclipse.core.databinding.observable.map.IObservableMap;
import org.eclipse.jface.databinding.viewers.ObservableListContentProvider;
import org.eclipse.jface.databinding.viewers.ObservableMapLabelProvider;
import org.eclipse.jface.viewers.TableViewer;
import org.eclipse.jface.viewers.TableViewerColumn;
import org.eclipse.swt.SWT;
import org.eclipse.swt.layout.GridData;
import org.eclipse.swt.layout.GridLayout;
import org.eclipse.swt.widgets.Composite;
import org.eclipse.swt.widgets.Table;
import org.eclipse.swt.widgets.TableColumn;
import org.eclipse.ui.part.ViewPart;

public class View extends ViewPart {
	public View() {
	}
	public static final String ID = "it.rcpvision.rcprap.application.view";
	private TableViewer viewer;
	private Table table;
	private WritableList writableList = new WritableList();
	private DataBindingContext m_bindingContext;

	public void createPartControl(Composite parent) {
		Library library = (Library) Activator.getDefault().getResource().getContents().get(0);
		writableList.addAll(library.getListAuthor());
		parent.setLayout(new GridLayout(1, false));

		viewer = new TableViewer(parent, SWT.BORDER | SWT.FULL_SELECTION);
		table = viewer.getTable();
		table.setLinesVisible(true);
		table.setHeaderVisible(true);
		table.setLayoutData(new GridData(SWT.FILL, SWT.FILL, true, true, 1, 1));

		TableViewerColumn tableViewerColumnName = new TableViewerColumn(viewer, SWT.NONE);
		TableColumn tblclmnName = tableViewerColumnName.getColumn();
		tblclmnName.setWidth(100);
		tblclmnName.setText("Name");

		TableViewerColumn tableViewerColumnSurname = new TableViewerColumn(viewer, SWT.NONE);
		TableColumn tblclmnSurname = tableViewerColumnSurname.getColumn();
		tblclmnSurname.setWidth(100);
		tblclmnSurname.setText("Surname");

		m_bindingContext = initDataBindings();
	}

	public void setFocus() {
	}
	protected DataBindingContext initDataBindings() {
		DataBindingContext bindingContext = new DataBindingContext();
		//
		ObservableListContentProvider listContentProvider = new ObservableListContentProvider();
		IObservableMap[] observeMap = PojoObservables.observeMaps(listContentProvider.getKnownElements(), Author.class, new String[]{"name", "surname"});
		viewer.setLabelProvider(new ObservableMapLabelProvider(observeMap));
		viewer.setContentProvider(listContentProvider);
		//
		viewer.setInput(writableList);
		//
		return bindingContext;
	}
}

you also need the following dependencies in MANIFEST.MF

Require-Bundle: org.eclipse.ui;resolution:=optional,
 org.eclipse.core.runtime,
 org.eclipse.rap.ui;resolution:=optional,
 it.rcpvision.rcptutorial.model,
 org.eclipse.emf.teneo.hibernate,
 org.hibernate,
 com.mysql.jdbc,
 org.eclipse.emf.ecore.xmi,
 org.eclipse.core.databinding,
 org.eclipse.core.databinding.beans,
 org.eclipse.core.databinding.observable,
 org.eclipse.core.databinding.property,
 org.eclipse.jface.databinding;resolution:=optional,
 org.eclipse.rap.jface.databinding;resolution:=optional

as you see we put other Optional dependencies on both org.eclipse.jface.databinding and org.eclipse.rap.jface.databinding.

If you now run this same source code from the RCP workspace you will see this

while from the RAP workspace you’ll see the following (Note: in both cases you may have to check that the launch has all needed plug-ins/bundles; see “Plug-ins” tab, “Add Required Plug-ins” button in case of RCP and tab “Bundles“, “Add Required Bundles” button in case of RAP)

not bad, don’t you think?

Back to index

RAP (Rich Ajax Platform) is the framework that allows you to develop an RCP application an make it available over the network just pointing the browser to a given link.
In this article we will see how you can develop an RCP application and launch it both as a desktop as well as a web application.  Yes, sure, exactly the same source code!
Don’t you believe it? Try it yourself.

Setup your IDE for RAP

The fastest way to get RAP working is downloading the Eclipse for RCP and RAP Developers package from page http://www.eclipse.org/downloads


Alternatively, you could add RAP from any other Eclipse Indigo SR2 (a.k.a. version 3.7.2) package, just going to Help -> Install New Software… -> select Indigo update site and install Rich Ajax Platform (RAP) Tooling  1.4.2 under the category Web, XML, Java EE and OSGi Enterprise Development.

Now we have to set up a RAP target plaftorm. Then start Eclipse and go to Window -> Preferences -> Plug-in Development -> Target Platform

and click Add… button and go Next

Give a name to the platform, e.g. RAP_1.4 and press Add… button in the first tab (Locations)

then select Software Site

and go Next. Then add the RAP 1.4 Runtime Update site
http://download.eclipse.org/rt/rap/1.4/runtime

uncheck that radio button Include required software and press Finish.

this may take quite a while. At the end that’s what you should see

click Finish.
Then follow the same procedure now adding, from Indigo update site  http://download.eclipse.org/releases/indigo, the feature EMF RAP Target Components from Category EclipseRT Target Platform Components. You should now see the following

The important point here is that now you have two target platforms: the RCP and RAP ones. Select the RAP one for this workspace and press OK.

Now start another Eclipse instance and choose a different workspace, that will be used for RCP development.
Is it becoming clear what we are going to do? We want to have two different workspaces, one for running the RCP (=Desktop) launch and the other for running the RAP (=Web) launch. But what about the project? The very same source code!

After starting Eclipse with the RCP workspace follow one of the wizard for creating an RCP application: File -> New -> Plug-in Project, give the plug-in a name (e.g. it.rcpvision.rcprap.application) and select Yes at question “Do you want to create a Rich Client Application?“. Then choose the second template “RCP application with a view” and press Finish. Now launch the application; you will see the following

Up to now nothing special, just what you probably do almost every day.

Now let’s do a simple change in the Dependencies of this plug-in. Select the dependency from org.eclipse.ui and make it Optional.

Then go the tab MANIFEST.MF and add a dependency from org.eclipse.rap.ui and make it Optional too.

If you run again the application you’ll get, of course, the same result. We just introduced a double, optional, dependency to both RCP and RAP.

Now go back to the other Eclipse instance, the one pointing to the workspace configured for RAP, and import the same project from the file system.
Then select Run -> Run Configurations… -> RAP Application -> New

In the first tab (Main) click the Browse button for field Entry Point or Application

and select our application

In the second folder (Bundles) press Deselect All and add our application (it.rcpvision.rcprap.application) and the following plug-ins

com.ibm.icu.base
javax.servlet
org.eclipse.core.commands
org.eclipse.core.contenttype
org.eclipse.core.databinding
org.eclipse.core.databinding.beans
org.eclipse.core.databinding.observable
org.eclipse.core.databinding.property
org.eclipse.core.expressions
org.eclipse.core.jobs
org.eclipse.core.runtime
org.eclipse.equinox.app
org.eclipse.equinox.common
org.eclipse.equinox.http.jetty
org.eclipse.equinox.http.registry
org.eclipse.equinox.http.servlet
org.eclipse.equinox.http.servletbridge
org.eclipse.equinox.preferences
org.eclipse.equinox.registry
org.eclipse.equinox.servletbridge
org.eclipse.equinox.servletbridge.extensionbundle
org.eclipse.help
org.eclipse.jdt.junit.runtime
org.eclipse.osgi
org.eclipse.osgi.services
org.eclipse.rap.jface
org.eclipse.rap.jface.databinding
org.eclipse.rap.junit
org.eclipse.rap.junit.runtime
org.eclipse.rap.rwt
org.eclipse.rap.rwt.q07
org.eclipse.rap.rwt.theme.classic
org.eclipse.rap.ui
org.eclipse.rap.ui.cheatsheets
org.eclipse.rap.ui.forms
org.eclipse.rap.ui.views
org.eclipse.rap.ui.workbench
org.junit
org.mortbay.jetty.server
org.mortbay.jetty.util

Note: if you don’t want to add them one by one, you can persist the launch (tab Common -> Save as -> Shared file) then open the launch as text and change the following line

with this content

<stringAttribute key=”target_bundles” value=”com.ibm.icu.base@default:default,javax.servlet@default:default,org.eclipse.core.commands@default:default,org.eclipse.core.contenttype@default:default,org.eclipse.core.databinding.beans@default:default,org.eclipse.core.databinding.observable@default:default,org.eclipse.core.databinding.property@default:default,org.eclipse.core.databinding@default:default,org.eclipse.core.expressions@default:default,org.eclipse.core.jobs@default:default,org.eclipse.core.runtime@default:true,org.eclipse.equinox.app@default:default,org.eclipse.equinox.common@2:true,org.eclipse.equinox.http.jetty@default:default,org.eclipse.equinox.http.registry@default:default,org.eclipse.equinox.http.servlet@default:default,org.eclipse.equinox.http.servletbridge@default:default,org.eclipse.equinox.preferences@default:default,org.eclipse.equinox.registry@default:default,org.eclipse.equinox.servletbridge.extensionbundle@default:false,org.eclipse.equinox.servletbridge@default:default,org.eclipse.help@default:default,org.eclipse.jdt.junit.runtime@default:default,org.eclipse.osgi.services@default:default,org.eclipse.osgi@-1:true,org.eclipse.rap.jface.databinding@default:default,org.eclipse.rap.jface@default:default,org.eclipse.rap.junit.runtime@default:default,org.eclipse.rap.junit@default:default,org.eclipse.rap.rwt.q07@default:false,org.eclipse.rap.rwt.theme.classic@default:default,org.eclipse.rap.rwt@default:default,org.eclipse.rap.ui.cheatsheets@default:default,org.eclipse.rap.ui.forms@default:default,org.eclipse.rap.ui.views@default:default,org.eclipse.rap.ui.workbench@default:default,org.eclipse.rap.ui@default:default,org.junit@default:default,org.mortbay.jetty.server@default:default,org.mortbay.jetty.util@default:default”/>

Well, we are ready: run the launch and you’ll see the internal browser opening your application as a Web application!

You can now copy the same URL on your preferred browser from the same machine, but of course, you can also do the same from another machine, just replace the IP (127.0.0.1) with the one that identifies your machine on the network.

Happy RAP to everyone!

Acknowledgments
Thanks to the RAP Team and to Hendy Irawan for this article
http://eclipsedriven.blogspot.it/2010/12/eclipse-rap-single-sourcing-awesomeness.html, which inspired this tutorial.

For further details on the event, visit the official website at www.eclipsedayflorence.com

The second edition of Codemotion, the event which is open to all languages and technologies will take place on 23rd and 24th March 2012 in Rome. The main innovations of Codemotion 2012 will be: two days dedicated to technical interventions and the fact that there will be two events held simultaneously in Rome and Madrid.

The lengthening from one to two days will be the occasion to arrange all the activities over a longer period of time. Participants will have the opportunity to listen to all talks without running the risk of having to deal with the overlapping of interventions.

Being held simultaneously, the events of Rome and Madrid will connect the two capital cities through passion for technology and for programming. Codemotion Roma and Codemotion Madrid are organized with the same spirit and format where the technical contents and Codemotion People will be the protagonists of the two events.

Codemotion will be held in Rome on 23rd and 24th March 2012.
Entry is free, as always.
For any further information: http://www.codemotion.it

In this tutorial we will create a simple RCP java product and then we’ll use Jubula testing tool to execute some automated tests. First we will create a minimal RCP application, using the Plug-in Project creation Eclipse Wizard. The next step will consist into setup the Jubula environment and the AUT Agent to link the application with jubula
The final step will be create some recorded tests and play them with Jubula. The result is a simple suite of tests that allow us to automatically run our tests saving our time. The main goal of this tutorial is show how to use Jubula testing tool with a RCP application.

Step 1: Setup Environment

First we have to setup the environment, downloading the Eclipse for Tester version: Eclipse for Testers (Indigo Packages): The Eclipse Jubula Download

And Jubula Standalone application from: The Jubula StandAlone Download

Step 2: Create the Plugin Project

The next step is create a new Plug-in Project. To do this, follow the steps showed in the figures below. From menu File select New then Plug-in Project

(File/New/Project…/Plug-in Development/Plug-in Project):

As Plug-in name enter “com.rcpvision.rcpmailsample”.

Next, tell Eclipse that it’s a rich client application:

And finally, choose the “RCP Mail” template:

Select “Yes” if you want to switch to the Plug-in development perspective.
You should now see the project MANIFEST.MF editor. It contains mainly the information about the dependencies from other projects and the exporting options. From here you can manage the project and launch it. Just click “Launch” as showed below.

Step 3: Create the Product

In order to use Jubula application you need to create a product configuration for the application just created. Starting from:
(File/New/Product Configuration)

Name it “SimpleMail.product”
To finish click on Finish button and in the com.rcpvision.rcpmailsample plugin will appear the SimpleMail.product file

Then export the Plugin Project:

The result should be a product like this one:

Step 4: Set and launch AUT Agent

The AUT Agent is the Jubula daemon providing the bridge between the test executor and the Application Under Test.
Once the AUT Agent has started the AUT, object mapping can take place, and tests can be observed and executed.

Step 4.1: Set AUT Agent

Launch “setup.exe” to install the AUT AGENT (Application Under Test Agent)

Accept the agreement then click on “Next”

click again “Next” and finally…..

Step 4.2: Launch AUT Agent

To execute AUT Agent under Windows you need administrator rights:

And you’ll see a little green icon  on your toolbar.

Step 4.3: Configuring the Product to work with Jubula

In order to work with Jubula under RCP mail product you need to extract the rcp-support.zip in your Jubula (AUT Agent) installation directory and copy it into your product directory (plugins folder) .
This will add a Jubula instrumentation plugin to your product. You’ve still got to activate that plugin, and you can do that by editing the configuration/config.ini file in your product directory.
There’s a long line in that file listing the osgi.bundles, and you have to add the Jubula plugin at the end:

And finally, after all these steps, you’re ready to work with Jubula!!!

Step 5: Working with Jubula

You are finally ready to open Jubula and create your first tests

Step 5.1: Start Jubula

Start Jubula:

Step 5.2: Create new Test Project

Looking at the problem view, you’ll find a hint about what to do next:

However, it doesn’t tell you how to create a new project. You will find it in the Test menu:
(Test/New…)
Give your project a good name, choose “rcp” as toolkit and click the “Next” button.

Pitfall: Make sure your test specifies either English (United States) or German (Germany)as the keyboard layout. These are the only two keyboard layouts provided out-of-the-box by Jubula at the moment.

Now you’ve got to tell Jubula about your test object (Jubula calls it “AUT”):
Give your AUT Agent a name and choose “rcp” as toolkit.

Click on “Proceed” to close the dialog and to continue working on New Project Wizard dialog.
Now you can choose to specify an AUT Configuration, by clicking on “Next” or click “Finish” and finish creating the project and configure the AUT later via “Project -> Properties”

Choosing to close the wizard menu, click on “Finish”.
You can notice that in the Test Browser view there is a new project “SimpleMailTest”

Step 5.3: Configuring AUT for Test Project

Focus on “Simple Test Mail” and click on
Test-> Properties

Click on “Edit”

Click on “Add”

Enter a suitable name for the AUT Agent Host and select the executable file of your product in the “Executable File Name”

Clicking on “Ok” we create the test project. Then click on the little arrow next to , choose the option “localhost: 600000″ to connect the AUT agent.

Now, look for the “Connect to AUT Agent” toolbar button and press it in order to establish the connection to the agent we started before:

Confirm

Click on Proceed and you’ll see your RCP Mail Template product

Step 6: Testing With Jubula

At the end of this section you’ll be able to create and run a set of test cases. Let’s test the opening of a new messages view.
There are three methods to open a new Message view and for all of these we’ll show you how to do.

Case 1: Using Menu Bar: File-> Open Another Message View Command

Case 2: Using toolbar button  

Case 3: Using shortcut: CTRL+2

Test Case 1

Goal of this test case is to replay the

File → Open Another Message View

operation recording jubula commands.
Preconditions: Application under test (RCP Product) running.

Steps:

From Jubula IDE click on the ‘camera’ icon

Name the test Case with a suitable name, i.e. “OpenNewMessageView”

and confirm your choice by clicking on “Ok” button.
Jubula will move into the “TestCase” perspective with the “OpenNewMessageView” tab open in the editor and a new item “OpenNewMessageView” listed under the “Test Case Browser” view.

After few seconds the “Observation Console” window will be opened in the bottom corner of your screen. In this windows will be listed all the steps done during registration. Now give the focus on the application under test (RCPMAIL) and Select “File-> Another Message View” option. You’ll create a new Message View in the RCPMail

and in the “Observation Console” will be added two new steps:

Stop the recording by clicking on “CameraStop Recording Icon”

Save the test case and in the Test Case perspective you’ll notice two steps under the your first test.
Now you have to create a new Test Suite: Within the “Test Suite Browser” view, right click on “Test Suites” folder and click on

New-> New Test Suite

Name it as “First Test Suite” and confirm (Click on “OK”)

Double click on First Test Suite just created and it will be opened in an editor. Now drag your use case “OpenNewMessageView” into the test suite and save the test project:

To play the test click on the little green icon in the “Test Suite Browser” then choose the “First Test Suite’

Test Case 2

Repeat previous steps to create a new Observed test case, record the steps to simulate the click on the green plus icon and the result will be:

Test Case 3

You have to create this test case by choosing a couple of template from the Test Case folder in the Test Case Browser. To do it follow these simple steps: From the folder “Test Cases” in “Test Case Browser” open “unbound_modules_concrete-> Actions (basic) → Input via Keyboard → Application → Key Combination” and select the test case named
<ub_app_pressAnyKeyCombination_CTRL[BASE_KEY]>
and the
<ub_app_pressAnyKey>[MODIFER_KEYS;BASE_KEY]>

the result should be:

Now you have to complete the settings for the test case just choosen.
For
<ub_app_pressAnyKeyCombination_CTRL[BASE_KEY]>

For
<ub_app_pressAnyKey>[MODIFER_KEYS;BASE_KEY]>

enter:
Parameter Name = MODIFIER_KEYS
Parameter Value = control

Parameter Name = BASE_KEY
Parameter Value = BASE_KEY

Test Suite

You can rename the first test case as “OpenNewMessageView_FileMenu”.
Drag and drop Test Case 2 and Test Case 3 to the Test Suite and you’ll have

Run the test suite and … Enjoy your automated test with Jubula!!!

In this post, I’d like to continue to inspect how to use only a small part of Xbase and still have the control on the generation part: in particular (for other projects) I would like to retain the control on the generation for my model, while relying on the Xbase generation for the Xbase parts. Thus, in this post I’ll describe:

• how to integrate Xbase variables (XVariableDeclaration) and expressions (XExpression) in your DSL
• how to extend Xbase scoping for making the variables visible in the expressions
• write a generator for your DSL and reuse the XbaseCompiler for the code of XVariableDeclaration and XExpressions

Following the spirit of the previous post, I won’t use the Domainmodel example, but something really simple, like the Greeting example, i.e., the very basic DSL you will get when creating an Xtext project inside Eclipse. In particular, you might want to go through the previous posts before reading this post (since we reuse some of the concepts seen there):

• the one about JVM types (concerning how to use the ImportManager for generating correct code for accessing Java types and the corresponding import statements)
• and the one about Xbase expressions (concerning how to use the XbaseCompiler for generating code corresponding to Xbase expressions).

So, first of all, create a new Xtext project; I’ll call this project org.xtext.example.helloxvars (and the files will have extension helloxvars). Now, we can start generate Xtext artifacts. With the defaults, you’ll also get a HelloXvarsGenerator.xtend, which we now to generate Java code. The basic idea of the generator is: for each Greeting element we will obtain a package and a class named according to the Greeting name feature (package all lower case, and class name with the first letter capital). We’ll deal with the generator later in this post.

Now, the idea is to enrich the greeting DSL with some Xbase variables (before the greeting statements) and some Xbase expressions (used within the greeting statements) which can access the declared variables; thus, we modify the grammar of the language (HelloXvars.xtext) as follows (you should be familiar with ‘import’ functionalities of Xtext):

grammar org.xtext.example.helloxvars.HelloXvars with
	org.eclipse.xtext.xbase.Xbase

generate helloXvars "http://www.xtext.org/example/helloxvars/HelloXvars"

Model:
	imports += Import*
	varDeclarations += XVariableDeclaration*
	greetings+=Greeting*;

Import:
	'import' importedNamespace = QualifiedNameWithWildcard
;

QualifiedNameWithWildcard:
	QualifiedName '.*'?
; 

Greeting:
	'Hello' name=ID 'from' expression = XExpression '!'
;

The idea is to end up (at the end of this post) with a DSL which allows us to write programs as follows:

Note that Xbase (its validator) already checks that the types are correct, i.e., that the initialization expression has a type which is a subtype of the declared type for the variable. But, more important, you will reuse the type inference mechanism of Xbase: you don’t need to declare the type for the variable when this can be inferred by Xbase itself!

Regenerate the Xtext artefacts, and start a new runtime eclipse instance. Create a new plugin project (say ‘helloxvars’), and in the source folder we create a .helloxvars file (say ‘My.helloxvars’) and accept to add the Xtext nature. To fully enjoy Xbase syntax, please make sure that in the project you created in the runtime workspace you also have org.eclipse.xtext.xbase.lib as a dependency in the MANIFEST.

Now, if you try to write code like in the above image you’ll get errors due to the Xbase expressions that cannot access Xbase variables; in fact, Xbase variables are automatically visible to Xbase expressions only when they are both part of an XBlockExpression, which we don’t use in our case. Thus, we must extend the Xbase scoping mechanism in order to make all the variables declared before the greetings to all the expressions of the greetings.

As soon as we start to use Xbase in our grammar, the generated runtime module will bind the IScopeProvider to XbaseScopeProvider; so, first of all, we must override this binding in our runtime module, in order to bind the scope provider to the one of our DSL:

public class HelloXvarsRuntimeModule extends
		org.xtext.example.helloxvars.AbstractHelloXvarsRuntimeModule {

	@Override
	public Class<? extends IScopeProvider> bindIScopeProvider() {
		return HelloXvarsScopeProvider.class;
	}

}

And now, we make our scope provider extend the XbaseScopeProvider; by inspecting the code of XbaseScopeProvider we find a possible good candidate method for customizing the scope of variables:

protected IScope createLocalVarScope(IScope parentScope,
			LocalVariableScopeContext scopeContext) {

Take a look at the code of this method in XbaseScopeProvider, and you should get the general idea (the base implementation walks through the containment hierarchy of the model to build the scope for local variables); in particular, LocalVariableScopeContext contains the context for the scope. In our case, we need to provide a custom scope when the context is the greeting Model (in all other cases we rely on the base implementation):

package org.xtext.example.helloxvars.scoping;

import org.eclipse.emf.ecore.EObject;
import org.eclipse.xtext.scoping.IScope;
import org.eclipse.xtext.scoping.Scopes;
import org.eclipse.xtext.xbase.scoping.LocalVariableScopeContext;
import org.eclipse.xtext.xbase.scoping.XbaseScopeProvider;
import org.xtext.example.helloxvars.helloXvars.Model;

/**
 * This class contains custom scoping description.
 *
 * see : http://www.eclipse.org/Xtext/documentation/latest/xtext.html#scoping on
 * how and when to use it
 *
 */
public class HelloXvarsScopeProvider extends XbaseScopeProvider {

	@Override
	protected IScope createLocalVarScope(IScope parentScope,
			LocalVariableScopeContext scopeContext) {
		if (scopeContext != null && scopeContext.getContext() != null) {
			EObject context = scopeContext.getContext();
			if (context instanceof Model) {
				Model model = (Model) context;
				return Scopes.scopeFor(model.getVarDeclarations());
			}
		}

		return super.createLocalVarScope(parentScope, scopeContext);
	}

}

So, when the context is the root of our model (Model) we return as the scope all the variables declared in our greeting model.

Now, restart your eclipse runtime instance and you’ll see that errors go away: the expressions in the greetings can access the variables declared above!

Note that the scoping does not concern only the expressions in the greetings, but also the initialization expressions of local variable declarations themselves:

val v1 = 'foo'
val v = new String(v1)
val v2 = v + v1

Now, if you do some experiments you’ll note that we provide scopes that contain too much! Try the following:

val v = new String(v2)
val v2 = v

This code should raise errors, since the initialization expression for v should not be able to access variables declared afterwards (e.g., v2); while, with our scope implementation, this code would be silently accepted.

So, we need to tweak our scope implementation; however, let’s stop following the workflow of trying a modification, restart the runtime eclipse instance, and see whether we made it… let’s be agile :) and write some unit tests! Create an Xtend2 class in the tests plugin project org.xtext.example.helloxvars.tests and add in the MANIFEST org.eclipse.xtext.xtend2.lib as a dependency.

package org.xtext.example.helloxvars.tests

import com.google.inject.Inject
import org.eclipse.xtext.diagnostics.Diagnostic
import org.eclipse.xtext.junit4.InjectWith
import org.eclipse.xtext.junit4.XtextRunner
import org.eclipse.xtext.junit4.util.ParseHelper
import org.eclipse.xtext.junit4.validation.ValidationTestHelper
import org.eclipse.xtext.xbase.XbasePackage
import org.junit.Test
import org.junit.runner.RunWith
import org.xtext.example.helloxvars.HelloXvarsInjectorProvider
import org.xtext.example.helloxvars.helloXvars.Model

@InjectWith(typeof(HelloXvarsInjectorProvider))
@RunWith(typeof(XtextRunner))
class HelloXvarsParserTest {

	@Inject
	ParseHelper<Model> parser

	@Inject extension ValidationTestHelper

	@Test
	def void testParsingAndLinking() {
		parser.parse("Hello foo from new String()!").assertNoErrors
	}

	@Test
	def void testParsingAndLinkingWithVars() {
		parser.parse("
			val s1 = 'foo'
			val s2 = 'bar'
			val s3 = s1 + s2
			Hello foo from new String(s3)!
		").assertNoErrors
	}

	@Test
	def void testParsingAndLinkingWithClosures() {
		parser.parse("
			val s1 = 'foo'
			val s2 = 'bar'
			val s3 = s1 + [ s | s.toFirstLower + s1 ].apply(s2 + s1)
			Hello foo from new String(s3)!
		").assertNoErrors
	}

	@Test
	def void testParsingAndLinkingWithMissingVar() {
		parser.parse("
			Hello foo from new String(s)!
		").assertError(XbasePackage::eINSTANCE.XFeatureCall,
				Diagnostic::LINKING_DIAGNOSTIC,
				"Couldn't resolve reference to JvmIdentifiableElement 's'.")
	}

	@Test
	def void testParsingAndLinkingWithWrongVarOrder() {
		parser.parse("
			val s1 = s2
			val s2 = s1
			Hello foo from new String(s1)!
		").assertError(XbasePackage::eINSTANCE.XFeatureCall,
				Diagnostic::LINKING_DIAGNOSTIC,
				"Couldn't resolve reference to JvmIdentifiableElement 's2'.")
	}

}

You see, the first test checks a greeting program without variables, the second checks a valid program with variable access, the third one checks a valid program with variable access and a closure, the forth one checks that accessing a variable which is not declared generates an error, and the last one checks that variable initialization expressions cannot access variable declared afterwards. Of course, the last test fails! And we need to get the green bar back now :)

So now we modify the scoping:

package org.xtext.example.helloxvars.scoping;

import java.util.List;

import org.eclipse.emf.ecore.EObject;
import org.eclipse.xtext.EcoreUtil2;
import org.eclipse.xtext.scoping.IScope;
import org.eclipse.xtext.scoping.Scopes;
import org.eclipse.xtext.xbase.XExpression;
import org.eclipse.xtext.xbase.XVariableDeclaration;
import org.eclipse.xtext.xbase.scoping.LocalVariableScopeContext;
import org.eclipse.xtext.xbase.scoping.XbaseScopeProvider;
import org.xtext.example.helloxvars.helloXvars.Model;

/**
 * This class contains custom scoping description.
 *
 * see : http://www.eclipse.org/Xtext/documentation/latest/xtext.html#scoping on
 * how and when to use it
 *
 */
public class HelloXvarsScopeProvider extends XbaseScopeProvider {

	@Override
	protected IScope createLocalVarScope(IScope parentScope,
			LocalVariableScopeContext scopeContext) {
		if (scopeContext != null && scopeContext.getContext() != null) {
			EObject context = scopeContext.getContext();
			Model containingModel = EcoreUtil2.getContainerOfType(context,
					Model.class);
			XVariableDeclaration containingVarDecl = EcoreUtil2
					.getContainerOfType(context, XVariableDeclaration.class);
			List<XExpression> varDeclarations = containingModel.getVarDeclarations();
			int index = varDeclarations.size();
			if (containingVarDecl != null) {
				index = varDeclarations.indexOf(containingVarDecl);
			}
			return Scopes.scopeFor(varDeclarations.subList(0, index));
		}

		return super.createLocalVarScope(parentScope, scopeContext);
	}

}

This time we do something different: the scope will still be all the variables of the containing greeting Model; but if we’re inside a XVariableDeclaration, the scope will be the list of all the variable declarations up to the current variable, and we get the green line back… oh oh… something it’s still wrong: the previous failing test now succeeds, but the test for closure now fails! It says it cannot resolve the reference to ‘s’, the local variable of the closure itself:

	@Test
	def void testParsingAndLinkingWithClosures() {
		parser.parse("
			val s1 = 'foo'
			val s2 = 'bar'
			val s3 = s1 + [ s | s.toFirstLower + s1 ].apply(s2 + s1)
			Hello foo from new String(s3)!
		").assertNoErrors
	}

If you take a look at our implementation of scoping, you’ll soon realize that we customized it too much: we basically override completely the method of XbaseScopeProvider, which will then be unable to build the scope for local variables declared in the expressions, and closures declare local variable themselves!

Indeed, the right solution is something in between the two implementations of scoping we saw here:

1. we must check whether the container of the context is a greeting Model
2. if it is, then the scope is the list of variables up to the current one, if the context is a variable declaration (we can simply search for the context in the list of variable declarations, since List.indexOf accept any Object
3. otherwise, we delegate to XbaseScopeProvider

package org.xtext.example.helloxvars.scoping;

import java.util.List;

import org.eclipse.emf.ecore.EObject;
import org.eclipse.xtext.scoping.IScope;
import org.eclipse.xtext.scoping.Scopes;
import org.eclipse.xtext.xbase.XExpression;
import org.eclipse.xtext.xbase.scoping.LocalVariableScopeContext;
import org.eclipse.xtext.xbase.scoping.XbaseScopeProvider;
import org.xtext.example.helloxvars.helloXvars.Model;

/**
 * This class contains custom scoping description.
 *
 * see : http://www.eclipse.org/Xtext/documentation/latest/xtext.html#scoping on
 * how and when to use it
 *
 */
public class HelloXvarsScopeProvider extends XbaseScopeProvider {

	@Override
	protected IScope createLocalVarScope(IScope parentScope,
			LocalVariableScopeContext scopeContext) {
		if (scopeContext != null && scopeContext.getContext() != null) {
			EObject context = scopeContext.getContext();
			if (context.eContainer() instanceof Model) {
				Model model = (Model) context.eContainer();
				List<XExpression> varDeclarations = model.getVarDeclarations();
				int index = varDeclarations.indexOf(context);
				if (index < 0)
					index = varDeclarations.size();
				return Scopes.scopeFor(varDeclarations.subList(0, index));
			}
		}

		return super.createLocalVarScope(parentScope, scopeContext);
	}

}

We finally get the green line for our tests :)

Now let’s move on to generation! Similarly to the previous post about Xbase expressions, we will generate a main method for each Greeting, and we will generate the Java code for all the variable declarations and initializations by reusing the XbaseCompiler. Most of the code for the generation has already been explained in the previous post.

package org.xtext.example.helloxvars.generator

import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IGenerator
import org.eclipse.xtext.generator.IFileSystemAccess
import com.google.inject.Inject
import org.eclipse.xtext.xbase.compiler.XbaseCompiler

import static extension org.eclipse.xtext.xbase.lib.IteratorExtensions.*
import org.xtext.example.helloxvars.helloXvars.Greeting
import org.eclipse.xtext.common.types.TypesFactory
import org.eclipse.xtext.xbase.compiler.ImportManager
import org.eclipse.xtext.xbase.XExpression
import org.eclipse.xtext.xbase.compiler.StringBuilderBasedAppendable
import org.xtext.example.helloxvars.helloXvars.Model
import org.eclipse.xtext.xbase.XFeatureCall

class HelloXvarsGenerator implements IGenerator {

	@Inject
	protected XbaseCompiler xbaseCompiler

	override void doGenerate(Resource resource, IFileSystemAccess fsa) {
		for(greeting: resource.allContents.toIterable.filter(typeof(Greeting))) {
			fsa.generateFile(
				greeting.packageName + "/" + // package
				greeting.className + ".java", // class name
				greeting.compile)
		}
	}

	def getJvmType(Greeting greeting) {
		val declaredType = TypesFactory::eINSTANCE.createJvmGenericType
		declaredType.setSimpleName(greeting.className)
		declaredType.setPackageName(greeting.packageName)
		declaredType
	}

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true, getJvmType(greeting))»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def compile(Greeting greeting, ImportManager importManager) {
		var result = new StringBuilderBasedAppendable(importManager)
		for (varDecl : (greeting.eContainer as Model).varDeclarations) {
			result.append("\n// variable declaration")
			result = compile(varDecl, result)
		}
		result.append("\n// greeting expression")
		result = compile(greeting.expression, result)
		var expressionVar = result.getName(greeting.expression)
		if (greeting.expression instanceof XFeatureCall)
			expressionVar = result.getName((greeting.expression as XFeatureCall).feature)
		val compiled = '''
		public class «greeting.className» {
			public static void main(String args[]) {
				«result»

				Object expression = «expressionVar»;
				System.out.println("Hello «greeting.name» from " +
					expression.toString());
			}
		}
		'''
		compiled
	}

	def compile(XExpression xExpression, StringBuilderBasedAppendable result) {
		xbaseCompiler.toJavaStatement(xExpression, result, true)
		result
	}

	def className(Greeting greeting) {
		greeting.name.toFirstUpper
	}

	def packageName(Greeting greeting) {
		greeting.name.toLowerCase
	}
}

Let us concentrate on the interesting part

def compile(Greeting greeting, ImportManager importManager) {
	var result = new StringBuilderBasedAppendable(importManager)
	for (varDecl : (greeting.eContainer as Model).varDeclarations) {
		result.append("\n// variable declaration")
		result = compile(varDecl, result)
	}
	result.append("\n// greeting expression")
	result = compile(greeting.expression, result)
	var expressionVar = result.getName(greeting.expression)
	if (greeting.expression instanceof XFeatureCall)
		expressionVar = result.getName((greeting.expression as XFeatureCall).feature)
	val compiled = '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«result»

			Object expression = «expressionVar»;
			System.out.println("Hello «greeting.name» from " +
				expression.toString());
		}
	}
	'''
	compiled
}

We basically use the XbaseCompiler for generating the Java code corresponding to variable declarations and initializations; then we need to access the generated Java variable name corresponding to the XExpression of the current Greeting element. We use the getName method, as explained in the previous post; however, this time, we need to distinguish the case when the XExpression is a XFeatureCall: this is the case when the XExpression is just the name of a declared variable, as in this code

var myList =
	new LinkedList<Integer>(Collections::singleton(10))

Hello barList from myList !

In this case, we need to get the variable name which corresponds to the feature of the XFeatureCall, and not to the XFeatureCall itself (since no synthetic expression is generated for the XFeatureCall itself). Let’s try the generator

Remember, that we did not want to generate something meaningful: we wanted to experiment with code generation :)

You can find the sources for the project helloxvars at

https://github.com/LorenzoBettini/Xtext2-experiments

Hope you find this post useful, and stay tuned for new posts about Xtext :)

If you take a look at the Five simple steps to your JVM language, you’ll see the power of Xbase! Moreover, in Xtext 2.1, using Xbase is even easier since by writing only a AbstractModelInferrer you’ll get a full integration with Java (by mapping your AST model elements to Java concepts) and a generator (which will produce Java code corresponding to this mapping). Thus, you won’t even need a generator at all.

However, in this post, I’d like to inspect how to use only a small part of Xbase and still have the control on the generation part: in particular (for other projects) I would like to retain the control on the generation for my model, while relying on the Xbase generation for the Xbase parts. Thus, in this post I’ll describe:

• how to integrate Xbase expressions (XExpression) in your DSL
• write a generator for your DSL and reuse the XbaseCompiler for the code of XExpressions

Following the spirit of the previous post, I won’t use the Domainmodel example, but something really simple, like the Greeting example, i.e., the very basic DSL you will get when creating an Xtext project inside Eclipse. In particular, you might want to go through the previous post before reading this post (since we reuse some of the concepts seen there).

So, first of all, create a new Xtext project; I’ll call this project org.xtext.example.helloxbase (and the files will have extension helloxbase). Now, we can start generate Xtext artifacts. With the defaults, you’ll also get a HelloXbaseGenerator.xtend, which we now to generate Java code. The first implementation of the generator is: for each Greeting element we will obtain a package and a class named according to the Greeting name feature (package all lower case, and class name with the first letter capital). Moreover, the generated main method will print the corresponding greeting:

package org.xtext.example.helloxbase.generator

import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IFileSystemAccess
import org.eclipse.xtext.generator.IGenerator
import org.xtext.example.helloxbase.helloXbase.Greeting

import static extension org.eclipse.xtext.xtend2.lib.ResourceExtensions.*

class HelloXbaseGenerator implements IGenerator {

	override void doGenerate(Resource resource, IFileSystemAccess fsa) {
		for(greeting: resource.allContentsIterable.filter(typeof(Greeting))) {
			fsa.generateFile(
				greeting.packageName + "/" + // package
				greeting.className + ".java", // class name
				greeting.compile)
		}
	}

	def compile(Greeting greeting) '''
	package «greeting.packageName»;

	public class «greeting.className» {
		public static void main(String args[]) {
			System.out.println("Hello «greeting.name»");
		}
	}
	'''

	def className(Greeting greeting) {
		greeting.name.toFirstUpper
	}

	def packageName(Greeting greeting) {
		greeting.name.toLowerCase
	}
}

If you’re already familiar with Xtend2 (or if you read the previous post), you should have a clue about what the generator does; let’s try the generator: restart your second eclipse instance, and make sure that in the plugin project you had created with the helloxbase file, you have a folder named src-gen, and that folder is configured as a source folder. If you type something like

Hello foo!
Hello bar!

you should see some Java code generated in the src-gen folder (according to the schema detailed above).

Now, the idea is to enrich the HelloXbase greeting DSL with some Xbase expressions, thus, we modify the grammar of the language (HelloXbase.xtext) as follows (you should be familiar with ‘import’ functionalities of Xtext):

grammar org.xtext.example.helloxbase.HelloXbase with
	org.eclipse.xtext.xbase.Xbase

generate helloXbase "http://www.xtext.org/example/helloxbase/HelloXbase"

Model:
	imports += Import*
	greetings+=Greeting*;

Import:
  'import' importedNamespace = QualifiedNameWithWildcard
;

QualifiedNameWithWildcard:
  QualifiedName '.*'?
; 

Greeting:
    'Hello' name=ID 'from' expression = XExpression '!'
;

Note that we now derive our grammar from the Xbase grammar, so we can now rely on Xbase grammar rules; the idea is to be able to write in our DSL something like

Try it yourself: regenerate the Xtext artefacts, and restart the runtime eclipse instance. To fully enjoy Xbase syntax, please make sure that in the project you created in the runtime workspace you also have org.eclipse.xtext.xbase.lib as a dependency in the MANIFEST.

You can see lots of cool stuff here! You basically have all the powerful Xbase expression Java-like syntax, access to static methods using ::, closures, and extension methods (the nullOrEmpty from StringExtensions)! All for free!

Now, what happened to our generator? Is it still invoked automatically on resource changes? I’m afraid not: as soon as you derived your grammar from Xbase, a new file appeared in your project, HelloXbaseJvmModelInferrer.xtend, since in Xtext 2.1, as hinted at the beginning, the generation strategy changed. In particular, in the src-gen AbstractHelloXbaseRuntimeModule the IGenerator is now bound to org.eclipse.xtext.xbase.compiler.JvmModelGenerator; since we want our own generator back, we simply override this binding in HelloXbaseRuntimeModule (the one we’re allowed to modify):

public class HelloXbaseRuntimeModule extends org.xtext.example.helloxbase.AbstractHelloXbaseRuntimeModule {

	/**
	 * Avoids to use the default org.eclipse.xtext.xbase.compiler.JvmModelGenerator
	 * when using xbase.
	 * @see org.xtext.example.helloxbase.AbstractHelloXbaseRuntimeModule#bindIGenerator()
	 */
	@Override
	public Class<? extends IGenerator> bindIGenerator() {
		return HelloXbaseGenerator.class;
	}

}

If you restart your runtime instance you’ll see our generator is back!

Now, we want to generate the Java code corresponding to the Xbase XExpression; we then simply reuse the XbaseCompiler in our generator (which, in Xtext, you know, means ‘have it injected’). In the generator we will now use also the ImportManager (which is explained in the previous post) and start to do some experiments with the XbaseCompiler:

package org.xtext.example.helloxbase.generator

import com.google.inject.Inject
import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.common.types.TypesFactory
import org.eclipse.xtext.generator.IFileSystemAccess
import org.eclipse.xtext.generator.IGenerator
import org.eclipse.xtext.xbase.XExpression
import org.eclipse.xtext.xbase.compiler.ImportManager
import org.eclipse.xtext.xbase.compiler.StringBuilderBasedAppendable
import org.eclipse.xtext.xbase.compiler.XbaseCompiler
import org.xtext.example.helloxbase.helloXbase.Greeting

import static extension org.eclipse.xtext.xtend2.lib.ResourceExtensions.*

class HelloXbaseGenerator implements IGenerator {

	@Inject
	protected XbaseCompiler xbaseCompiler

	override void doGenerate(Resource resource, IFileSystemAccess fsa) {
		for(greeting: resource.allContentsIterable.filter(typeof(Greeting))) {
			fsa.generateFile(
				greeting.packageName + "/" + // package
				greeting.className + ".java", // class name
				greeting.compile)
		}
	}

	def getJvmType(Greeting greeting) {
		val declaredType = TypesFactory::eINSTANCE.createJvmGenericType
		declaredType.setSimpleName(greeting.className)
		declaredType.setPackageName(greeting.packageName)
		declaredType
	}

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true, getJvmType(greeting))»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«compile(greeting.expression, importManager)»

			System.out.println("Hello «greeting.name» from ");
		}
	}
	'''

	def compile(XExpression xExpression, ImportManager importManager) {
		val result = new StringBuilderBasedAppendable(importManager)
		xbaseCompiler.toJavaStatement(xExpression, result, true)
		result
	}

	def className(Greeting greeting) {
		greeting.name.toFirstUpper
	}

	def packageName(Greeting greeting) {
		greeting.name.toLowerCase
	}
}

Let’s concentrate on these parts:

	def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«compile(greeting.expression, importManager)»

			System.out.println("Hello «greeting.name» from ");
		}
	}
	'''

	def compile(XExpression xExpression, ImportManager importManager) {
		val result = new StringBuilderBasedAppendable(importManager)
		xbaseCompiler.toJavaStatement(xExpression, result, true)
		result
	}

We tried to use the toJavaStatement of XbaseCompiler, which seems a good candidate for what we want; in particular, by passing true as the last argument we say that the “result” of generation should be assigned to a Java variable, so that we can refer to it; let’s see what happens in the runtime workspace with this generation:

Hey, cool! The XbaseCompiler generates all the Java statements corresponding to the original XExpression (including closures)! Now we would only like to access the last variable generated (in the examples above, _isNullOrEmpty and _apply, respectively), so that we can generate some Java code ourselves using that variable; but how can we know its name?

Well, the StringBuilderBasedAppendable class not only contains the result of the generation; it also contains a map with all the intermediate Java local variables generated by the compiler. Each variable, in the map, has as the key the corresponding Xbase object. So we only need to know the variable name corresponding to our XExpression and we’re done:

result.getName(greeting.expression)

So the final generator looks like this:

package org.xtext.example.helloxbase.generator

import com.google.inject.Inject
import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.common.types.TypesFactory
import org.eclipse.xtext.generator.IFileSystemAccess
import org.eclipse.xtext.generator.IGenerator
import org.eclipse.xtext.xbase.XExpression
import org.eclipse.xtext.xbase.compiler.ImportManager
import org.eclipse.xtext.xbase.compiler.StringBuilderBasedAppendable
import org.eclipse.xtext.xbase.compiler.XbaseCompiler
import org.xtext.example.helloxbase.helloXbase.Greeting

import static extension org.eclipse.xtext.xtend2.lib.ResourceExtensions.*

class HelloXbaseGenerator implements IGenerator {

	@Inject
	protected XbaseCompiler xbaseCompiler

	override void doGenerate(Resource resource, IFileSystemAccess fsa) {
		for(greeting: resource.allContentsIterable.filter(typeof(Greeting))) {
			fsa.generateFile(
				greeting.packageName + "/" + // package
				greeting.className + ".java", // class name
				greeting.compile)
		}
	}

	def getJvmType(Greeting greeting) {
		val declaredType = TypesFactory::eINSTANCE.createJvmGenericType
		declaredType.setSimpleName(greeting.className)
		declaredType.setPackageName(greeting.packageName)
		declaredType
	}

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true, getJvmType(greeting))»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def compile(Greeting greeting, ImportManager importManager) '''
	«val result = compile(greeting.expression, importManager)»
	public class «greeting.className» {
		public static void main(String args[]) {
			«result»

			Object expression = «result.getName(greeting.expression)»;
			System.out.println("Hello «greeting.name» from " +
				expression.toString());
		}
	}
	'''

	def compile(XExpression xExpression, ImportManager importManager) {
		val result = new StringBuilderBasedAppendable(importManager)
		xbaseCompiler.toJavaStatement(xExpression, result, true)
		result
	}

	def className(Greeting greeting) {
		greeting.name.toFirstUpper
	}

	def packageName(Greeting greeting) {
		greeting.name.toLowerCase
	}
}

And in the runtime instance you’ll get what you wanted:

You can find the sources for the project helloxbase at

https://github.com/LorenzoBettini/Xtext2-experiments

Hope you find this post useful, and stay tuned for new posts about Xtext :)

This paragraph in the documentation briefly describe how to refer to Java elements using JVM Types, and then dedicates much more room to accessing Java Types using Xbase.

In this post, I’d like to document some my experiments/experiences in accessing JVM Types without using Xbase, and in particular, not by using the good ol’ Domainmodel example, but something even more simpler, so that I could concentrate on the issue of using JVM Types (and not with more involved features of the DSL itself).

Thus, for this simple experiment, I’ll use, as the starting point, the Greeting example, i.e., the very basic DSL you will get when creating an Xtext project inside Eclipse. So, go on and create such project (see the strings I’ve used in the screenshot):

The defaults for the created project are already fine for accessing JVM Types, so we don’t have to tweak the mwe2 file.

Now, following the Xtext documentation, we import the ecore package defining JVM Types into our HelloJvmTypes.xtext

import "http://www.eclipse.org/xtext/common/JavaVMTypes" as jvmTypes

and we are now ready for accessing JvmTypes from our grammar.

I don’t want to have a useful DSL, I just want to experiment with accessing Java types, so let’s say I just want to end up with a DSL that lets me write sentences like

Hello world from java.util.List, org.eclipse.emf.ecore.EClass!
Hello foobar from java.io.IOException!

and from these sentences, generate some Java classes that simply print the same strings.

So we change our HelloJvmTypes.xtext as follows

grammar org.xtext.example.hellojvmtypes.HelloJvmTypes with org.eclipse.xtext.common.Terminals

import "http://www.eclipse.org/xtext/common/JavaVMTypes" as jvmTypes 

generate helloJvmTypes "http://www.xtext.org/example/hellojvmtypes/HelloJvmTypes"

Model:
 greetings+=Greeting*;

Greeting:
 'Hello' name=ID 'from'
 javaTypes+=[jvmTypes::JvmType|QualifiedName]
 (',' javaTypes+=[jvmTypes::JvmType|QualifiedName])* '!'
;

QualifiedName: ID ('.' ID)* ;

Now we can regenerate all the artefacts, and run another Eclipse instance; here we create a new plugin project (say ‘hellojvmtypes’), and in the source folder we create a .hellojvmtypes file (say ‘My.hellojvmtypes’). We should end up with the editor as in the screenshot

Note that also the content assist for Java types works! (if you didn’t type anything where a JvmType is expected and you ask for content assist you might experience some delays because the proposals consist of all the visible Java types in your project).

Note also that only the Java types which are “reachable” from your project’s classpath will be actually visible. For instance, try to use org.eclipse.emf.ecore.EClass and you’ll get an error (if you created a simple plugin project).

Now, try and add org.eclipse.emf.ecore as a dependency in you project MANIFEST file and you’ll then be able to access its Java classes.

Before we go on with code generation, let’s do some unit testing! Create an Xtend2 class in the tests plugin project org.xtext.example.hellojvmtypes.tests and add in the MANIFEST org.eclipse.xtext.xtend2.lib as a dependency.

package org.xtext.example.hellojvmtypes.tests

import com.google.inject.Inject
import junit.framework.Assert
import org.eclipse.xtext.common.types.JvmGenericType
import org.eclipse.xtext.junit4.InjectWith
import org.eclipse.xtext.junit4.XtextRunner
import org.eclipse.xtext.junit4.util.ParseHelper
import org.eclipse.xtext.junit4.validation.ValidationTestHelper
import org.junit.Test
import org.junit.runner.RunWith
import org.xtext.example.hellojvmtypes.HelloJvmTypesInjectorProvider
import org.xtext.example.hellojvmtypes.helloJvmTypes.Greeting
import org.xtext.example.hellojvmtypes.helloJvmTypes.Model

@InjectWith(typeof(HelloJvmTypesInjectorProvider))
@RunWith(typeof(XtextRunner))
class ParserTest {

    @Inject
    ParseHelper<Model> parser

    @Inject extension ValidationTestHelper

    @Test
    def void testParsingAndLinking() {
        parser.parse('''Hello foo from java.util.List!''').assertNoErrors
    }

    @Test
    def void testJvmTypeAccess() {
        val model = parser.parse(
            "Hello foo from java.util.List!")
        val greeting = model.greetings.head as Greeting
        val jvmType = greeting.javaTypes.get(0) as JvmGenericType
        Assert::assertEquals("java.util.List", jvmType.identifier)
    }

}

If you run the corresponding generated Java class as a Junit test you’ll see the green line!

Now, let’s write the generator, by modifying the xtend2 class which Xtext had already created for you in your project:

package org.xtext.example.hellojvmtypes.generator

import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IGenerator
import org.eclipse.xtext.generator.IFileSystemAccess
import org.xtext.example.hellojvmtypes.helloJvmTypes.Greeting

import static extension org.eclipse.xtext.xtend2.lib.ResourceExtensions.*
import org.eclipse.xtext.xbase.compiler.ImportManager

class HelloJvmTypesGenerator implements IGenerator {

	override void doGenerate(Resource resource, IFileSystemAccess fsa) {
		for(greeting: resource.allContentsIterable.filter(typeof(Greeting))) {
			fsa.generateFile(
				greeting.packageName + "/" + // package
				greeting.className + ".java", // class name
				greeting.compile)
		}
	}

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true)»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«FOR javaType : greeting.javaTypes»
			System.out.println("Hello «greeting.name» from " +
				«importManager.serialize(javaType)».class.getName());
			«ENDFOR»
		}
	}
	'''

	def className(Greeting greeting) {
		greeting.name.toFirstUpper
	}

	def packageName(Greeting greeting) {
		greeting.name.toLowerCase
	}
}

Before explaining the code above, let’s try the generator: restart your second eclipse instance, and make sure that in the project you had created with the hellojvmtypes file you have a folder named src-gen, and that folder is configured as a source folder.

You should now see the code generated automatically in src-gen: in particular you will end up, for each Greeting element with a package and a class named according to the Greeting name feature (package all lower case, and class name with the first letter capital). Moreover, the generated main method will print the corresponding greeting.

The important parts in the generator are the following ones:

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true)»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«FOR javaType : greeting.javaTypes»
			System.out.println("Hello «greeting.name» from " +
				«importManager.serialize(javaType)».class.getName());
			«ENDFOR»
		}
	}
	'''

Here we make use of a really cool class provided by Xtext for generation of import statements and in particular for generation of Java class accesses: quoting from Xtext documentation:

The ImportManager shortens fully qualified names, keeps track of imported namespaces, avoids name collisions

Thus, if you call importManager.serialize(JvmType) you will not only have a string for the passed JvmType: ImportManager will keep track of an import statement which will have to be added to guarantee that the generated string results in valid Java code. Of course, in case of conflicts the ImportManager will generate a fully qualified Java name (and no import statement will be recorded). Thus, in our code generator we first generate the code for the class and the main method and save it into a String;

def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true)»
	«val mainMethod = compile(greeting, importManager)»
... continued in (2)

during the generation the importManager recorded the required imports,

def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«FOR javaType : greeting.javaTypes»
			System.out.println("Hello «greeting.name» from " +
				«importManager.serialize(javaType)».class.getName());
			«ENDFOR»
		}
	}
	'''

thus, we then generate all the Java import statements, and then the actual Java class (that we buffered into a string).

(2)... continuation
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

For instance, see how the ImportManager correctly (and transparently) handles possible Java class conflicts in the generated code (due to the class URI which appears in different packages).

So far, so good! But what happens if we refer to a Java type which, by chance, has the same class name of the class we’re generating? For instance, in the src folder create a class hello.Foobar and refer to it in your My.hellojvmtypes and see the generated code… argh! We get an error!

That is because the ImportManager knows about the types you’re accessing in the generation of the class Foobar (in this case), but not about the generated class Foobar itself! We can solve this problem by using the other form of ImportManager constructor

public ImportManager(boolean organizeImports, JvmDeclaredType thisType)

where you specify the JvmDeclaredType of the Java element which will contain the accesses to Java elements we are generating through the ImportManager itself. Thus, we only need to create on the fly a JvmDeclaredType corresponding to the Java class we are generating for the Greeting element!

Here’s the modification to the generator:

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true, createJvmType(greeting))»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def createJvmType(Greeting greeting) {
	    val declaredType = TypesFactory::eINSTANCE.createJvmGenericType
	    declaredType.setSimpleName(greeting.className)
	    declaredType.setPackageName(greeting.packageName)
	    declaredType
	}

now, restart the other eclipse instance, and regenerate the code for My.hellojvmtypes, and see now the correct generated Java class!

So, this is the complete final version of the generator:

package org.xtext.example.hellojvmtypes.generator

import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IGenerator
import org.eclipse.xtext.generator.IFileSystemAccess
import org.xtext.example.hellojvmtypes.helloJvmTypes.Greeting

import static extension org.eclipse.xtext.xtend2.lib.ResourceExtensions.*
import org.eclipse.xtext.xbase.compiler.ImportManager
import org.eclipse.xtext.common.types.TypesFactory

class HelloJvmTypesGenerator implements IGenerator {

	override void doGenerate(Resource resource, IFileSystemAccess fsa) {
		for(greeting: resource.allContentsIterable.filter(typeof(Greeting))) {
			fsa.generateFile(
				greeting.packageName + "/" + // package
				greeting.className + ".java", // class name
				greeting.compile)
		}
	}

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true, createJvmType(greeting))»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def createJvmType(Greeting greeting) {
	    val declaredType = TypesFactory::eINSTANCE.createJvmGenericType
	    declaredType.setSimpleName(greeting.className)
	    declaredType.setPackageName(greeting.packageName)
	    declaredType
	}

	def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«FOR javaType : greeting.javaTypes»
			System.out.println("Hello «greeting.name» from " +
				«importManager.serialize(javaType)».class.getName());
			«ENDFOR»
		}
	}
	'''

	def className(Greeting greeting) {
		greeting.name.toFirstUpper
	}

	def packageName(Greeting greeting) {
		greeting.name.toLowerCase
	}
}

You can find the sources for the project hellojvmtypes at

https://github.com/LorenzoBettini/Xtext2-experiments

You can find another post on Xtext and Xbase here Xtext 2.1: using Xbase expressions.

Hope you find this post useful, and stay tuned for new posts about Xtext :)