Using JVM Types in Xtext 2.1 and the ImportManager

Xtext 2.1 made it even easier to access Java types from your DSL; you can find some paragraphs in the documentation. In particular, the new features of Xbase seem to make this integration even more powerful!

This paragraph in the documentation briefly describe how to refer to Java elements using JVM Types, and then dedicates much more room to accessing Java Types using Xbase.

In this post, I’d like to document some my experiments/experiences in accessing JVM Types without using Xbase, and in particular, not by using the good ol’ Domainmodel example, but something even more simpler, so that I could concentrate on the issue of using JVM Types (and not with more involved features of the DSL itself).

Thus, for this simple experiment, I’ll use, as the starting point, the Greeting example, i.e., the very basic DSL you will get when creating an Xtext project inside Eclipse. So, go on and create such project (see the strings I’ve used in the screenshot):

The defaults for the created project are already fine for accessing JVM Types, so we don’t have to tweak the mwe2 file.

Now, following the Xtext documentation, we import the ecore package defining JVM Types into our HelloJvmTypes.xtext

import "http://www.eclipse.org/xtext/common/JavaVMTypes" as jvmTypes

and we are now ready for accessing JvmTypes from our grammar.

I don’t want to have a useful DSL, I just want to experiment with accessing Java types, so let’s say I just want to end up with a DSL that lets me write sentences like

Hello world from java.util.List, org.eclipse.emf.ecore.EClass!
Hello foobar from java.io.IOException!

and from these sentences, generate some Java classes that simply print the same strings.

So we change our HelloJvmTypes.xtext as follows

grammar org.xtext.example.hellojvmtypes.HelloJvmTypes with org.eclipse.xtext.common.Terminals

import "http://www.eclipse.org/xtext/common/JavaVMTypes" as jvmTypes 

generate helloJvmTypes "http://www.xtext.org/example/hellojvmtypes/HelloJvmTypes"

Model:
 greetings+=Greeting*;

Greeting:
 'Hello' name=ID 'from' 
 javaTypes+=[jvmTypes::JvmType|QualifiedName]
 (',' javaTypes+=[jvmTypes::JvmType|QualifiedName])* '!'
;

QualifiedName: ID ('.' ID)* ;

Now we can regenerate all the artefacts, and run another Eclipse instance; here we create a new plugin project (say ‘hellojvmtypes’), and in the source folder we create a .hellojvmtypes file (say ‘My.hellojvmtypes’). We should end up with the editor as in the screenshot

Note that also the content assist for Java types works! (if you didn’t type anything where a JvmType is expected and you ask for content assist you might experience some delays because the proposals consist of all the visible Java types in your project).

Note also that only the Java types which are “reachable” from your project’s classpath will be actually visible. For instance, try to use org.eclipse.emf.ecore.EClass and you’ll get an error (if you created a simple plugin project).

Now, try and add org.eclipse.emf.ecore as a dependency in you project MANIFEST file and you’ll then be able to access its Java classes.

Before we go on with code generation, let’s do some unit testing! Create an Xtend2 class in the tests plugin project org.xtext.example.hellojvmtypes.tests and add in the MANIFEST org.eclipse.xtext.xtend2.lib as a dependency.

package org.xtext.example.hellojvmtypes.tests

import com.google.inject.Inject
import junit.framework.Assert
import org.eclipse.xtext.common.types.JvmGenericType
import org.eclipse.xtext.junit4.InjectWith
import org.eclipse.xtext.junit4.XtextRunner
import org.eclipse.xtext.junit4.util.ParseHelper
import org.eclipse.xtext.junit4.validation.ValidationTestHelper
import org.junit.Test
import org.junit.runner.RunWith
import org.xtext.example.hellojvmtypes.HelloJvmTypesInjectorProvider
import org.xtext.example.hellojvmtypes.helloJvmTypes.Greeting
import org.xtext.example.hellojvmtypes.helloJvmTypes.Model

@InjectWith(typeof(HelloJvmTypesInjectorProvider))
@RunWith(typeof(XtextRunner))
class ParserTest {

    @Inject
    ParseHelper<Model> parser

    @Inject extension ValidationTestHelper

    @Test
    def void testParsingAndLinking() {
        parser.parse('''Hello foo from java.util.List!''').assertNoErrors
    }

    @Test
    def void testJvmTypeAccess() {
        val model = parser.parse(
            "Hello foo from java.util.List!")
        val greeting = model.greetings.head as Greeting
        val jvmType = greeting.javaTypes.get(0) as JvmGenericType
        Assert::assertEquals("java.util.List", jvmType.identifier)        
    }

}

If you run the corresponding generated Java class as a Junit test you’ll see the green line!

Now, let’s write the generator, by modifying the xtend2 class which Xtext had already created for you in your project:

package org.xtext.example.hellojvmtypes.generator

import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IGenerator
import org.eclipse.xtext.generator.IFileSystemAccess
import org.xtext.example.hellojvmtypes.helloJvmTypes.Greeting

import static extension org.eclipse.xtext.xtend2.lib.ResourceExtensions.*
import org.eclipse.xtext.xbase.compiler.ImportManager

class HelloJvmTypesGenerator implements IGenerator {

	override void doGenerate(Resource resource, IFileSystemAccess fsa) {
		for(greeting: resource.allContentsIterable.filter(typeof(Greeting))) {
			fsa.generateFile(
				greeting.packageName + "/" + // package
				greeting.className + ".java", // class name
				greeting.compile)
		}
	}

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true)»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«FOR javaType : greeting.javaTypes»
			System.out.println("Hello «greeting.name» from " +
				«importManager.serialize(javaType)».class.getName());
			«ENDFOR»
		}
	}
	'''

	def className(Greeting greeting) {
		greeting.name.toFirstUpper
	}

	def packageName(Greeting greeting) {
		greeting.name.toLowerCase
	}
}

Before explaining the code above, let’s try the generator: restart your second eclipse instance, and make sure that in the project you had created with the hellojvmtypes file you have a folder named src-gen, and that folder is configured as a source folder.

You should now see the code generated automatically in src-gen: in particular you will end up, for each Greeting element with a package and a class named according to the Greeting name feature (package all lower case, and class name with the first letter capital). Moreover, the generated main method will print the corresponding greeting.

The important parts in the generator are the following ones:

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true)»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«FOR javaType : greeting.javaTypes»
			System.out.println("Hello «greeting.name» from " +
				«importManager.serialize(javaType)».class.getName());
			«ENDFOR»
		}
	}
	'''

Here we make use of a really cool class provided by Xtext for generation of import statements and in particular for generation of Java class accesses: quoting from Xtext documentation:

The ImportManager shortens fully qualified names, keeps track of imported namespaces, avoids name collisions

Thus, if you call importManager.serialize(JvmType) you will not only have a string for the passed JvmType: ImportManager will keep track of an import statement which will have to be added to guarantee that the generated string results in valid Java code. Of course, in case of conflicts the ImportManager will generate a fully qualified Java name (and no import statement will be recorded). Thus, in our code generator we first generate the code for the class and the main method and save it into a String;

def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true)»
	«val mainMethod = compile(greeting, importManager)»
... continued in (2)

during the generation the importManager recorded the required imports,

def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«FOR javaType : greeting.javaTypes»
			System.out.println("Hello «greeting.name» from " +
				«importManager.serialize(javaType)».class.getName());
			«ENDFOR»
		}
	}
	'''

thus, we then generate all the Java import statements, and then the actual Java class (that we buffered into a string).

(2)... continuation
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

For instance, see how the ImportManager correctly (and transparently) handles possible Java class conflicts in the generated code (due to the class URI which appears in different packages).

So far, so good! But what happens if we refer to a Java type which, by chance, has the same class name of the class we’re generating? For instance, in the src folder create a class hello.Foobar and refer to it in your My.hellojvmtypes and see the generated code… argh! We get an error!

That is because the ImportManager knows about the types you’re accessing in the generation of the class Foobar (in this case), but not about the generated class Foobar itself! We can solve this problem by using the other form of ImportManager constructor

public ImportManager(boolean organizeImports, JvmDeclaredType thisType)

where you specify the JvmDeclaredType of the Java element which will contain the accesses to Java elements we are generating through the ImportManager itself. Thus, we only need to create on the fly a JvmDeclaredType corresponding to the Java class we are generating for the Greeting element!

Here’s the modification to the generator:

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true, createJvmType(greeting))»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def createJvmType(Greeting greeting) {
	    val declaredType = TypesFactory::eINSTANCE.createJvmGenericType
	    declaredType.setSimpleName(greeting.className)
	    declaredType.setPackageName(greeting.packageName)
	    declaredType
	}

now, restart the other eclipse instance, and regenerate the code for My.hellojvmtypes, and see now the correct generated Java class!

So, this is the complete final version of the generator:

package org.xtext.example.hellojvmtypes.generator

import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IGenerator
import org.eclipse.xtext.generator.IFileSystemAccess
import org.xtext.example.hellojvmtypes.helloJvmTypes.Greeting

import static extension org.eclipse.xtext.xtend2.lib.ResourceExtensions.*
import org.eclipse.xtext.xbase.compiler.ImportManager
import org.eclipse.xtext.common.types.TypesFactory

class HelloJvmTypesGenerator implements IGenerator {

	override void doGenerate(Resource resource, IFileSystemAccess fsa) {
		for(greeting: resource.allContentsIterable.filter(typeof(Greeting))) {
			fsa.generateFile(
				greeting.packageName + "/" + // package
				greeting.className + ".java", // class name
				greeting.compile)
		}
	}

	def compile(Greeting greeting) '''
	«val importManager = new ImportManager(true, createJvmType(greeting))»
	«val mainMethod = compile(greeting, importManager)»
	package «greeting.packageName»;
	«IF !importManager.imports.empty»

	«FOR i : importManager.imports»
		import «i»;
	«ENDFOR»
	«ENDIF»

	«mainMethod»
	'''

	def createJvmType(Greeting greeting) {
	    val declaredType = TypesFactory::eINSTANCE.createJvmGenericType
	    declaredType.setSimpleName(greeting.className)
	    declaredType.setPackageName(greeting.packageName)
	    declaredType
	}

	def compile(Greeting greeting, ImportManager importManager) '''
	public class «greeting.className» {
		public static void main(String args[]) {
			«FOR javaType : greeting.javaTypes»
			System.out.println("Hello «greeting.name» from " +
				«importManager.serialize(javaType)».class.getName());
			«ENDFOR»
		}
	}
	'''

	def className(Greeting greeting) {
		greeting.name.toFirstUpper
	}

	def packageName(Greeting greeting) {
		greeting.name.toLowerCase
	}
}

You can find the sources for the project hellojvmtypes at

https://github.com/LorenzoBettini/Xtext2-experiments

You can find another post on Xtext and Xbase here Xtext 2.1: using Xbase expressions.

Hope you find this post useful, and stay tuned for new posts about Xtext :)

This entry was tagged , . Bookmark the permalink.

2 Responses to Using JVM Types in Xtext 2.1 and the ImportManager

  1. Nice post!

    One remark: initializing variables (like importManager and mainMethod) inside expressions in rich strings seems a bit over the top. I think it’s much clearer if you do that just before the rich string itself.

  2. Bettini says:

    Hi Meinte, I’m glad you enjoyed the post! :)

    To be honest, I based that generation part on the one found in Xtext documentation, http://www.eclipse.org/Xtext/documentation/2_1_0/199b-xbase-java-references.php :)

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>