One of the most common tasks when working with models is executing queries on them (e.g. finding a set of corresponding model elements for code generation or finding violations for model constraints) – quite often repeatedly. To allow effective calculation and recalculation, an index of the EMF model has to be created and maintained during editing operations.
EMF-IncQuery is a tool we are developing at the Budapest University of Technology and Economics try to solve this issue by providing a runtime library for these indexes, and additionally an Eclipse-based tooling to specify and debug such queries. I have already blogged about a validation framework based on this technology, and my collegue, István Ráth gave a short presentation in last year’s EclipseCon Europe Modeling Symposium.
However, previous tool demos were held using a tooling and query language originally created for the VIATRA2 model transformation framework, so it was somewhat hard to use. Since these demonstrations we created a new, Xtext-based tooling using a modified query language that fits the EMF model specifications better.
The Xtext-based editor with type information hovers.
Another new user interface component is the Query Explorer: a view that allows evaluating developed queries without using the generated code in a new Eclipse instance, while allowing the reuse of existing domain editors.
The Query Explorer can be used to evaluate patterns using existing model editors.
As for examples, since last year we developed two new case studies for EMF-IncQuery:
We also created connectors for JFace databinding for query results allowing query-backed automatically updating user interfaces.
Even better, this new version is available since Wednesday – for a detailed release notes see the announcement post in our homepage. If you want to download this release, you can use the Eclipse Marketplace client (at least if search is working as expected – today we managed to not find our software there using the built-in search 🙁 ). Alternatively, EMF-IncQuery is available from our update site: http://viatra.inf.mit.bme.hu/update/incquery/
Alltogether, the new EMF-IncQuery release marks an important point: we believe at this point, it is ready for use. For me, this is an important checkpoint, as in the last year I put a considerable effort of getting this rolling.
As for the future, EMF-IncQuery is on its track to becoming an Eclipse project: the creation review is scheduled for the 10th October 2012. After the creation, we plan to migrate our codebase, and have of course further ideas on how to improve the system to be usable in more and more cases.
Xtext is a great technology. I really mean it. After 15-30 minutes (depending on your previous domain knowledge) you can get a really fancy textual editor, that parses your text into an EMF model. Xbase is even better: you can simply include Java-like expressions into your language, that can be easily translated into pure Java code.
However, while using Xtext I found some minor “paper-cuts”. One of the most problematic of them was the fact that I cannot open my textual models using the Sample Reflective Ecore Model editor, as I get an exception: java.net.MalformedURLException: unknown protocol: java.
A similar exception occurred if I tried to open the files programatically (but without the help of the generated Xtext editor). When I asked in the Eclipse forums how to open Xtext model files programatically, I was answered (btw. very quickly – kudos for that 🙂 ) that a specific ResourceSet implementation is needed that refers to model files and classes available in a Java project.
This gave me the idea to create an Xtext extension that allows opening an EMF tree editor, the Xtext Reflective Tree Editor. The plug-in consists of two parts:
And an Xtext generator fragment, that can be used to generate a new editor extension to the ui project (with the corresponding dependency added to the reflective editor plug-in).
Usage
To use the plug-in, just follow the following simple steps:
Add hu.cubussapiens.xtext.reflectiveeditor as a dependency to your language project.
Extend your editor generation workflow:
Import the generator package: import hu.cubussapiens.xtext.reflectiveeditor.generator.*
Add the new generator fragment at the end of the generator (file.extensions is the list of file extensions defined in the default workflow): fragment = ReflectiveXtextEditorFragment {
fileExtensions = file.extensions
}
Regenerate your language.
Unless the plugin.xml files are updated automatically, you have to copy the newly generated editor extension from the end of the plugin.xml_gen file of the ui plug-in to the plugin.xml file. The simplest way to do this is to select both files, and open the compare editor from the popup menu (Compare with/Each other…), the copy the changes manually.
Finally, open the runtime workbench, and open your textual model files from the popup menu using Open with/«LanguageName» Reflective Editor. The result should be similar to the screenshot below.
The Xtext Reflective Editor
In my case, the editor shows both the domain model generated from the file directly (the Pattern Model element), the inferred JVM model classes (the various JVM Generic Type instances), and also the referred Java classes as external resources (the resources beginning with the java:/ URIs).
Download
If you are interested in the plug-in in action, download it from our update site: http://eclipse.cubussapiens.hu, or take a look at the repository on Github: https://github.com/ujhelyiz/xtext-reflective
Version 0.5.3 is already available, and hopefully works well for all languages.
If you’re developing with EMF it is a common task to separate the model into multiple files along with enabling the user to create crosslinks between them. This works out-of box if the files are located inside the same project. However, in some cases for larger models it is simply not enough. It can be useful to be able to define reusable (and versioned) packets of models and enable the user to define a configuration which determines the imported model packages. Sounds familiar? The same functionality exists in PDE. Maybe it can be reused for non-java purposes.
EMF provides a nice code generation facility for generating Java class hierarchy from ecore models. There is support for hand-made modifications to the generated code, or a large selection of generation parameters are available through a generator model.
However, we had a different need together with the VIATRA2 project: we explicitly forbid in our internal policy modifications to the generated Java code in order to avoid non-reusable models, and as this way the generated code does not contain any extra information to the generator model, we do not want to store the Java classes in our Subversion repository.
Another minor obstacle was that the code generation does not happen automatically (e.g. as in the LPG editor of the IMP project) – so simply not committing the generated code to the repository would mean a manual invocation of the generator that is inconvenient.
Luckily, the code generator is available using specific ANT tasks, allowing to write a build file that describes how to generate the required Java code, reusing the existing build model if necessary:
As the selected Ant task is provided by an OSGi plug-in, it is important to set the Runtime JRE to the same one as used by the workspace, otherwise the called task will not be found.
As the genmodel generator is not available as an Ant task, the ecore2Java task was used. In this case it is important to use the reconcilegenmodel attribute to avoid the unnecessary override of the genmodel
Regardless of the value of the generate*project attributes if the *project were not given as sub-tags, the corresponding code was not generated. This seems like a bug for me, as this duplicates information already present in the genmodel, but I am not sure.
The code generator does not set the generated files Derived attribute, thus it is not excluded from the svn repository. Unfortunately, it is not possible without writing custom tasks, as there is no such command (yet) in Eclipse (but hopefully the patch gets committed in Indigo).
This ant task mimics the GUI-based code generators functionality, thus fulfilling our needs, except two things: it does not exclude generated code from the repository, and does not run automatically.
The repository exclusion is resolved using a hack: the src folders are listed in all generated projects in the svn:ignore variable – thus manually disallowing their sharing. This approach works well, because if no code is generated, the src folder might be missing, that is marked with a very explicit error message (as it is still referenced in the project properties), but if the task is executed, the folder is created as well (and the source folder reference will be working).
For the automatic building Eclipse provides a neat way: External tools (e.g. Ant scripts, or shell scripts, etc.) can be added as project builders. A Run configuration has to be created for the selected task, that can be added to the project. This means, our script is executed each time the selected project changes
Builders page in the Project properties
Finally, some other modifications were made to the task in order to provide a more standard build: first, as not the entire project is generated at every invocation of the script (the projects could not be deleted, because the generator does not support setting the version number of the generated edit and editor projects), if all project stubs are downloaded from the repository, the build order might not be adequate. To overcome this, a manual build is executed for all projects at the end.
A second modification is a clean task, that removes all source folders, in order to provide a “clean”, out-of-the-box experience for a regeneration. This task can also be registered at the builder properties.
The full, modified script looks as following (using Gist from Github):
Update: the eclipse.build tasks are not available in Eclipse, as mentioned by a comment. They are provided by the Buckminster tool, and are not required for the EMF code generation, so it is safe to remove them. However, in this case the refreshing and building of the projects have to be managed in another way (e.g. registering the Ant script as an External builder for the projects).
This simple script helps in a lot of cases – and because the ecore models does not change very often, its performance is not really an issue (we have quite a large ecore model, so the generator runs about 30-40 seconds). My main issues are the following:
I don’t like to reproduce information from the genmodel in the build script. I am not entirely sure which information is read from the genmodel and which is inferred from the parameters, making the solution a bit volatile.
The missing derived flag command. It would be quite a nice touch if the code would take care of it.
Alltogether, if you like it, feel free to use it (licensed under EPL). I also would be glad if somebody could give me a hint about how to enhance the script – this is only my second Ant build script, so I don’t know the language very well. 🙂
Today I tried to edit an Ecore/Generator model used in the VIATRA2 framework, but the system greated me with an error screen stating, there are 4 errors in the selected Genmodel.
The error messages were a little bit cryptic (but all similar in nature): A containment reference of a type with a container feature platform:/resource/org.eclipse.viatra2.gtasm.model/model/gtasmmodel.ecore#//gtasm/metamodel/gt/GTPattern/namespace that requires instances to be contained elsewhere cannot be populated.
This was scary, as I don’t remember any new core features in EMF 2.6, that suggest such changes. My second guess was a misedited model, but after checking, that I have the same models as half year ago, this proved wrong. Even worse, these errors did not appear in either the Problems or Error log view. Checking the items mentioned in the error messages did not help, as the model elements looked “innocent enough” not to question their content.
It was only know when I saw that these errors are shown in a new, problem page in the editor (I often miss the editor pages option – except on form-based editors), and on the other page the tree is displayed, and code generation seemed also working.
At this point I looked on the internet for similar problems, and found a thread about the issue in the EMF newsgroup. The main idea was Ed Merks’s comment:
The point is supposed to be that you can’t be contained by more than one container, so you can’t have a container references that’s required as well as any other containment reference that’s not the opposite of that required container. I.e., a required container prevents other containment references from ever being populated.
Using this information I finally understood, what the error message meant, and begin looking for the cause in the Ecore model. After some searching (the error message only mentioned one model element from the three actors, and it is hard to get an overview of the inter modelelement relations in an EMF tree editor), I finally found the mentioned elements, that looked quite innocent at first – just take a look at the included graphical representation.
Illustration of the found problem
We have a machine, that contains both patterns and rules, while rules may contain addition pattern definitions (named local pattern definitions). On the other hand, patterns and rules have a backreference to the machine (namespace). As we wanted to have the Machine reference available for local patterns, the pattern references exactly one machine.
The drawback of the solution is, that the machine reference is the inverse of the containment relation, so every pattern has to be contained in a machine. On the other hand, EMF requires that every EObject (not EClass) must have exactly one incoming containment reference for serialization, so it is not possible for a rule to be the container of a pattern – hence the error message.
This analysis is extremely useful: it shows, that our metamodel prevents having serializable models in some cases, which comes in handy (I wouldn’t like to be the one, who has to debug such an error after a runtime exception is caught…). The fix is not really straightforward, as this model is already used (interesting fact, that the issue has not been produced in runtime – this means, we don’t have 100% test coverage :)), but by having a map to the error it is easier to fix.
I’d like to thank for the EMF team for this fine analysis solution, but some minor additions: don’t add warnings to a new page, use the existing reporting options; and try to rephrase the message for it to become more understandable.
