Source code visualisation for my projects

Posted on by
Reply
Today I read the announcement of SourceCloud, a new Zest/Cloudio-based visualization for source code. As I like cool visualizations, I tried it out on the open projects I am participating in.
At first I created the cloud for the source code of the Debug Visualisation project. Globally it consists of about half Eclipse debugger/UI and Java keywords – as this is quite a small project (about 4 kLOC), this is somewhat expected with 200 words.

SourceCloud of the Debug Visualisation Project

However, it is interesting, that the strings ‘end_of_line‘ and ‘sp_cleanup‘ found their way in. Luckily, only a few one-character names and numbers are present, meaning, we managed to created longer variable names…

The cloud of the VIATRA2 model transformation framework is more interesting: it is a much larger codebase, developed for several year by around a dozen people. Because of the size I generated the cloud using 300 words, but almost no Java keywords found their way into the graph.

SourceCloud of the VIATRA2 framework

However, the mandatory comment parts did, e.g. nbsp or the names of some creators. A lot of EMF-specific keywords are also present. They are present because we have quite a large EMF metamodel that provides several hundred generated Java files.

At last, but not least I also checked the cloud of the EMF-IncQuery project. This is smaller, then the VIATRA2 project, but larger then the Debug Visualisation. Additionally, it has a shorter history, but with several key committers.

SourceCloud of the EMF-IncQuery project

My personal favorite is this diagram, as it shows a lot of nice things: e.g. four digit numbers in large quantities (magic numbers FTW) or the absolute champion string: ‘HUKfyM7ahfg‘. All of them are present in the Ecore Diagram of the defined metamodel (the meaningless string 152 times as a postfix of an identifier :) ). At least, I learned something about the internal model representation of GMF.

And how do your projects look like?

Incremental validation of UML models using EMF-IncQuery

Posted on by
1

Last week we created in the university a nice demo application to demonstrate the validation capabilities of our EMF-IncQuery framework. The application was presented on the conference ECMFA 2011 as a tutorial.

EMF-IncQuery provides an incrementally evaluated model query framework: as the underlying EMF model is changed, the query results are re-evaluated, and are instantly returned when needed. Queries are formulated using the pattern language of the VIATRA2 transformation system, and the Java code necessary to execute queries are generated.

In our demo application we extended the Papyrus UML editor with on-the-fly validation of well-formedness constraints. During editing we check for non-local constraints (e.g. a Behaviour must have the same number of parameters as the corresponding operation); display the results in the editor and in the problems view; we try to provide and remove these error markers without saving or manual need to start validation.

Basically, the constraint validators are generated from specially annotated VIATRA2 patterns; the generation is currently somewhat specific to the Papyrus editor, but can be extended to support most EMF-based editors easily.

A screencast of the demo is presented below. It is important to note, that the Papyrus editor was not modified in any way, and the framework generated all Java code used during validation. More details about the demo application, such as source code, tutorial slides, can be found on the homepage of the application.

(If the embedded video does not show – e.g. in a feed reader – you could watch it on on Youtube)

Custom markers and annotations – the bright side of Eclipse

Posted on by
6

Motto:

I have discovered a truly remarkable proof of this theorem which this margin is too small to contain.
Pierre de Fermat

During the development of editors it is often needed to display additional information next to the document, such as error markers, search results or various coverage metrics. However, it is not a good idea to hardcode the list of supported markers during development, as the best ideas will be introduced by others – who do not want to have anything to do with our editor.

Luckily, Eclipse provides a simple way to attach markers to documents without explicit support needed from the editor developer: the markers (and annotations) created by their corresponding extensions are stored independently from the document, so these additions can be made without disturbing any existing user of the files.

As I described it in my previous post about error markers, such markers can be defined creating an extension for the extension point org.eclipse.core.resources.marker, and then using the IResource.createMarker() method to create the marker instances as needed. Then using some magic it can be displayed as the yellow or red wriggly lines during the text.

Other markers (and annotations) can be created in similarly: we create our own marker type, and register an annotation type for it, and the platform manages the rest. In my case I wanted to represent a calculated set of program statements similar to the code coverage display used by the EclEmma tool: the covered statements are highlighted with the green background.

To support such a display, I created a new marker type, that is a text marker; and to make the lifecycle-management easier, I did not set it persistent. This has been reached using the following code:

<extension
    id="org.eclipse.viatra2.slicemarker"
    name="GTASM Slice"
    point="org.eclipse.core.resources.markers">
  <super
    type="org.eclipse.core.resources.textmarker">
  </super>
</extension>

The most important difference between this marker and the one defined in the previous Markers and Annotations post is that the newly defined marker is not a problem marker, but a simple text marker. This means, the created marker will not show in the Problems view (that is nice – we don’t want to display problems now), and no default annotation is created (that will need fixing).

In theory it is possible to create annotations programmatically together with the markers, but in practice it is really a bad idea. First of all, annotations are created for the document model (thus assuming an open editor and a code dependency to that document model), are never persisted, and the resulting code fails indeterministically (I once debugged a code like that, every annotation creation code ran correctly, but some annotations were missing; re-running the code re-added the missing ones, but sometimes removed others), so this idea will not work.

Luckily, the platform provides two extension points that can be used to automatically assign annotations to marker instances. To define an automatically generated annotation type for the markers, the org.eclipse.ui.editors.annotationTypes extension point is used: basically a marker and an annotation ID is stored this way.

 

   <extension
         point="org.eclipse.ui.editors.annotationTypes">
      <type
            markerType="org.eclipse.viatra2.slicemarker"
            name="org.eclipse.viatra2.slicemarker">
      </type>
   </extension>

 

To define the appearance of the created annotation the org.eclipse.ui.editors.markerAnnotationSpecification extension point can be used. In the extensions we have to specify the annotation display methods used together with color and formatting settings.

 

   <extension
         point="org.eclipse.ui.editors.markerAnnotationSpecification">
      <specification
            annotationType="org.eclipse.viatra2.slicemarker"
            colorPreferenceKey="org.eclipse.viatra2.slice.color"
            colorPreferenceValue="192,255,192"
            contributesToHeader="false"
            highlightPreferenceKey="org.eclipse.viatra2.slice.highlight"
            highlightPreferenceValue="true"
            includeOnPreferencePage="true"
            label="GTASM Slice Marker"
            overviewRulerPreferenceKey="org.eclipse.viatra2.slice.overview"
            overviewRulerPreferenceValue="true"
            presentationLayer="0"
            textPreferenceKey="org.eclipse.viatra2.slice.text"
            textPreferenceValue="true"
            textStylePreferenceValue="BOX"
            verticalRulerPreferenceKey="org.eclipse.viatra2.slice.ruler"
            verticalRulerPreferenceValue="true">
      </specification>
   </extension>

Even better, by assigning preference keys to the various settings the platform also offers the users the possibility to configure the annotations: the General/Editors/Text Editors/Annotations preference page will list our annotation, and allows setting its preference values. If the preference keys are unique (that is our responsibility :) ), then creating this extension ends our work: annotations will be generated automatically, and they will be displayed in the opened text editors.

Annotation Preferences

The marker creation process is entirely the same as the creation of error markers: a new marker is initialized using the IResource.createMarker(String markerId) method, and the IMarker.CHAR_START and IMarker.CHAR_END markers are set for positioning the markers.

An editor opened with the annotated code

Alltogether, the markers and annotations provide an easy way to extend existing editors with meta-information (e.g. execution state, dependencies, coverage, etc.), that can be used with a very little amount of Java programming. Sadly, the content assists for extensions is not that discoverable as the Java APIs, so a lot of experimenting and/or documentation reading is needed. Luckily, documentation is available for these extension points, so knowing their name will be enough for most cases.

So, lets hope, Eclipse gives us a big enough margin to contain our proo metainformation.

How I recorded my part for the Virtual Choir

Posted on by
1

When I heard of Eric Whitacre’s Virtual Choir, I was immediately sold. After a lot bit of procrastination, I finally got to recording my part. However, things went not as easy as I had thought.

The first obstacle I encountered was that my MacBook’s built-in microphone is extremely noisy, so I had to record the video and audio separately, since there’s anyway a conductor track to synchronize to (so the video and audio won’t shift out of sync, there will be only temporary discrepancies), and I will attempt the recording multiple times anyway.

The next unpleasant surprise: playing a video consumes so much CPU that the fan is turned on, which is way too noisy. This was also a showstopper. I would have tried CPU throttling, but unfortunately cputhrottle didn’t work: the binary archive was corrupted, and the source couldn’t in any way compile. I almost gave up, when the LG GT540 Android smartphone I just got for this Christmas came into mind: it could serve perfectly as the video player!

This also wasn’t easy as pie, though. After downloading the video with Video DownloadHelper and copying it to the SD card, Android’s pre-bundled Media Player couldn’t open it. I found the excellent arcMedia player, which could play it, but at an impossibly slow frame rate. So I had to resize it: I chose HandBrake, and with the preset iPod conversion profile selected (320×192), the downgraded video played smoothly even in the built-in Media Player.

So I finally grabbed the phone, my Yamaha Pocketrak 2G recorder and the printed sheet music (I preferred the SATB score to the part), and dwelved into the music. (Of course, our neighbor’s dog barked into the silence at the end – I already expected that, but I managed to cut it out.)

However, replacing the audio track of the video was also hampered: iMovie ’08 can’t import the clip it recorded to a project, so I had to Google again. After another fair amount of time, I found SimpleMovieX, which suited my needs. I separated the video & audio track, added the new audio track to the video, synchronized them a bit and at last, the dubbed version was ready.

You can view the result here. I’m glad to finally be able to join a worldwide choir in this innovative social media experiment.