The research of my group often build on various Eclipse technologies. We are developers of the VIATRA2 model transformation framework. Recently we also created EMF-IncQuery, an incremental query technology over EMF models, that we believe really useful in various areas, such as model validation during editing or model synchronization scenarios.
This year, a collegue of mine, István Ráth will attend at EclipseCon Europe, and during the Modeling Symposium present the basic use cases of the tool. In the following I attach a short abstract of the talk to give a basic idea:
EMF-IncQuery: Incremental evaluation of model queries over large EMF model
A Talk at the EclipseCon Europe 2011 Modeling Symposium Presenter: István Ráth
High-performance model queries are still a major challenge for the industry standard Eclipse Modeling Framework (EMF), as they are intensively used in various model validation, model transformation or code generation scenarios.
Existing EMF-based query technologies (like OCL, EMF Query or native Java programs) have significant scalability issues for models over 50000 model elements. Moreover, it is often tedious and time consuming to efficiently implement EMF-based queries manually on a case-by-case basis.
This talk introduces EMF-IncQuery, a declarative and scalable EMF model query framework. EMF-IncQuery uses a graph query language, and provides incremental query evaluation by caching the results of the model queries and incrementally maintaining the cache when the underlying EMF model changes. Furthermore, the EMF-IncQuery framework can be easily integrated into existing EMF-based applications in a non-intrusive way.
During the talk, we quickly overview how easy it is to define and integrate highly scalable model queries into existing EMF-based applications, in the form of a very short live demonstration using the MDT Papyrus modeling tool. The scalability of the engine will also be demonstrated, with on-the-fly constraint revalidation that takes less than 100 milliseconds over large AUTOSAR models with over 1 million elements.
More info: http://viatra.inf.mit.bme.hu/incquery
Unfortunately, I cannot go to the conference, but I hope, the 10th birthday of Eclipse will be celebrated well. Have fun, and if you are interested, listen at our talk.
Today I read the announcement of SourceCloud, a new Zest/Cloudio-based visualization for source code. As I like cool visualizations, I tried it out on the open projects I am participating in.At first I created the cloud for the source code of the Debug Visualisation project. Globally it consists of about half Eclipse debugger/UI and Java keywords – as this is quite a small project (about 4 kLOC), this is somewhat expected with 200 words.
SourceCloud of the Debug Visualisation Project
However, it is interesting, that the strings ‘end_of_line‘ and ‘sp_cleanup‘ found their way in. Luckily, only a few one-character names and numbers are present, meaning, we managed to created longer variable names…
The cloud of the VIATRA2 model transformation framework is more interesting: it is a much larger codebase, developed for several year by around a dozen people. Because of the size I generated the cloud using 300 words, but almost no Java keywords found their way into the graph.
SourceCloud of the VIATRA2 framework
However, the mandatory comment parts did, e.g. nbsp or the names of some creators. A lot of EMF-specific keywords are also present. They are present because we have quite a large EMF metamodel that provides several hundred generated Java files.
At last, but not least I also checked the cloud of the EMF-IncQuery project. This is smaller, then the VIATRA2 project, but larger then the Debug Visualisation. Additionally, it has a shorter history, but with several key committers.
SourceCloud of the EMF-IncQuery project
My personal favorite is this diagram, as it shows a lot of nice things: e.g. four digit numbers in large quantities (magic numbers FTW) or the absolute champion string: ‘HUKfyM7ahfg‘. All of them are present in the Ecore Diagram of the defined metamodel (the meaningless string 152 times as a postfix of an identifier 🙂 ). At least, I learned something about the internal model representation of GMF.
Last week we created in the university a nice demo application to demonstrate the validation capabilities of our EMF-IncQuery framework. The application was presented on the conference ECMFA 2011 as a tutorial.
EMF-IncQuery provides an incrementally evaluated model query framework: as the underlying EMF model is changed, the query results are re-evaluated, and are instantly returned when needed. Queries are formulated using the pattern language of the VIATRA2 transformation system, and the Java code necessary to execute queries are generated.
In our demo application we extended the Papyrus UML editor with on-the-fly validation of well-formedness constraints. During editing we check for non-local constraints (e.g. a Behaviour must have the same number of parameters as the corresponding operation); display the results in the editor and in the problems view; we try to provide and remove these error markers without saving or manual need to start validation.
Basically, the constraint validators are generated from specially annotated VIATRA2 patterns; the generation is currently somewhat specific to the Papyrus editor, but can be extended to support most EMF-based editors easily.
A screencast of the demo is presented below. It is important to note, that the Papyrus editor was not modified in any way, and the framework generated all Java code used during validation. More details about the demo application, such as source code, tutorial slides, can be found on the homepage of the application.
(If the embedded video does not show – e.g. in a feed reader – you could watch it on on Youtube)
I have discovered a truly remarkable proof of this theorem which this margin is too small to contain.
Pierre de Fermat
During the development of editors it is often needed to display additional information next to the document, such as error markers, search results or various coverage metrics. However, it is not a good idea to hardcode the list of supported markers during development, as the best ideas will be introduced by others – who do not want to have anything to do with our editor.
Luckily, Eclipse provides a simple way to attach markers to documents without explicit support needed from the editor developer: the markers (and annotations) created by their corresponding extensions are stored independently from the document, so these additions can be made without disturbing any existing user of the files.
As I described it in my previous post about error markers, such markers can be defined creating an extension for the extension point org.eclipse.core.resources.marker, and then using the IResource.createMarker() method to create the marker instances as needed. Then using some magic it can be displayed as the yellow or red wriggly lines during the text.
Other markers (and annotations) can be created in similarly: we create our own marker type, and register an annotation type for it, and the platform manages the rest. In my case I wanted to represent a calculated set of program statements similar to the code coverage display used by the EclEmma tool: the covered statements are highlighted with the green background.
To support such a display, I created a new marker type, that is a text marker; and to make the lifecycle-management easier, I did not set it persistent. This has been reached using the following code:
The most important difference between this marker and the one defined in the previous Markers and Annotations post is that the newly defined marker is not a problem marker, but a simple text marker. This means, the created marker will not show in the Problems view (that is nice – we don’t want to display problems now), and no default annotation is created (that will need fixing).
In theory it is possible to create annotations programmatically together with the markers, but in practice it is really a bad idea. First of all, annotations are created for the document model (thus assuming an open editor and a code dependency to that document model), are never persisted, and the resulting code fails indeterministically (I once debugged a code like that, every annotation creation code ran correctly, but some annotations were missing; re-running the code re-added the missing ones, but sometimes removed others), so this idea will not work.
Luckily, the platform provides two extension points that can be used to automatically assign annotations to marker instances. To define an automatically generated annotation type for the markers, the org.eclipse.ui.editors.annotationTypes extension point is used: basically a marker and an annotation ID is stored this way.
To define the appearance of the created annotation the org.eclipse.ui.editors.markerAnnotationSpecification extension point can be used. In the extensions we have to specify the annotation display methods used together with color and formatting settings.
Even better, by assigning preference keys to the various settings the platform also offers the users the possibility to configure the annotations: the General/Editors/Text Editors/Annotations preference page will list our annotation, and allows setting its preference values. If the preference keys are unique (that is our responsibility 🙂 ), then creating this extension ends our work: annotations will be generated automatically, and they will be displayed in the opened text editors.
Annotation Preferences
The marker creation process is entirely the same as the creation of error markers: a new marker is initialized using the IResource.createMarker(String markerId) method, and the IMarker.CHAR_START and IMarker.CHAR_END markers are set for positioning the markers.
An editor opened with the annotated code
Alltogether, the markers and annotations provide an easy way to extend existing editors with meta-information (e.g. execution state, dependencies, coverage, etc.), that can be used with a very little amount of Java programming. Sadly, the content assists for extensions is not that discoverable as the Java APIs, so a lot of experimenting and/or documentation reading is needed. Luckily, documentation is available for these extension points, so knowing their name will be enough for most cases.
So, lets hope, Eclipse gives us a big enough margin to contain our proo metainformation.
I update the Debug Visualisation plug-in – a nasty bug was present related to opening new nodes that caused in the best case slowdowns, in worst cases exceptions and crashes.
This release is a bugfix release only, it is avaiable from the update site or Eclipse Marketplace, and recommended for every user of the plug-in.
