Nothing terrible actually, let’s take the example of a contribution to the p2 project (part of the equinox umbrella at the Eclipse foundation)

• Checkout the project

git clone http://git.eclipse.org/gitroot/equinox/rt.equinox.p2.git

• do the changes and then commit them to your local repo (you can perform this step several times)

git commit

• create the patch against the master branch

git format-patch origin

You  will get a file for each commit you submitted, so one commit =  one patch file.

You can attach your patch(es) to a bugzilla entry, for example , make p2 buildable with tycho (click on one of the patches)

• what an eclipse committer will do then , is apply your patch to its local repo to test everything is ok

git apply 0001-my-patch.patch

That’s pretty much it ! A bit more tedious than a Github pull request, but it does the job !

Mon objectif premier est de réaliser des sauvegardes robustes, la cerise sur le gâteau (ou sundae) est le fait de pouvoir consulter mes fichiers multimédias sur mon téléviseur (via une vieille XBOX avec XBMC et/ou une PS3)

Architecture de sauvegarde robuste

On réalise des sauvegardes, car on ne veut pas perdre des années de photo, de travail sur documents multimédia (montages vidéos) ses documents personnels (impôts, etc..).

La majorité des utilisateurs sauvegarde leurs fichiers vers un disque dur externe, quand ils y pensent…

Cette approche là est résolument insatisfaisante : il faut automatiser (fréquence) le processus de sauvegarde (donc avoir du matériel toujours disponible pour effectuer la sauvegarde) et aussi diversifier ses lieux de sauvegarde (tout sauvegarder sur des supports qui se trouvent au même endroit que les originaux ouvre la porte à une perte de donnés en cas d’inondation, incendie, cambriolage, etc…)

Je vais décrire ici comment utiliser un NAS Synology pour :

• y sauvegarder ses documents (les machines du réseau local, dans notre cas sous Ubuntu, vont sauvegarder vers le NAS)
• synchroniser ces sauvegardes vers une machine distante (le NAS va synchroniser les sauvegardes vers une machine dans un data center)
• y sauvegarder des sites internet et boîtes courriels hébergés sur un serveur distant (une machine dans un data center va sauvegarder ses données vers le NAS)

Enfin, je décrirai quelques astuces pour profiter de son NAS en serveur multimédia.

Sauvegarde entre les ordinateurs locaux et le NAS

Il existe un excellent article qui introduit ce propos sur le wiki de Synology; ceci dit la technique est la même que celle que j’avais décrit pour des sauvegardes distantes.

Mon objectif est ici de sauvegarder mon home directory (/home/anthony en excluant tous les .*) et rien d’autre
Voici mes remarques et ajouts :

• Depuis l’interface graphique de votre NAS, rendez vous dans Panneau de configuration>Utilisateur, sélectionnez votre utilisateur et appuyez et cocher « Accueil Utilisateur » (une traduction sympathique de Home directory :-p) , vous disposez maintenant d’un répertoire personnel dans /volume1/homes/votre_utilisateur
• la première étape est la création d’une clef (DSA, c’est important, RSA n’a pas marché chez moi…) sans mot de passe; utilisez la série de commande standard suivantes, dans mon cas :

ssh-keygen -t dsa -b 1024 -f ~/.ssh/diskstation

(pas de mot de passe, il serait difficile de gérer des sauvegardes non interactives avec un mot de passe, vérifiez quand même que votre clef .ssh/diskstation est en 600)

• Maintenant envoyer la clef publique à votre NAS :

ssh-copy-id -i .ssh/diskstation.pub anthony@diskstation

• Tester la connexion à votre NAS en utilisant la clef : (attention, il est fort possible que votre utilisateur n’aie pas de shell utilisable par défaut; connectez vous alors via SSH en tant que root (même mot de passe que admin) à votre NAS et éditez /etc/passwd, sur la ligne correspondant à votre utilisateur, remplacez /sbin/login par /bin/ash ; l’interpréteur de commandes par défaut)

ssh -i .ssh/diskstation diskstation

vous devriez arriver, sans mot de passe, dans le répertoire /volume1/homes/votre_utilisateur de votre NAS.

• rsync pour sauvegarder, la commande est assez simple :

rsync -avz --delete /home/anthony/ --exclude=.* diskstation:/volume1/homes/anthony/sauvegarde

• sauf que le vrai défi avec les sauvegardes, c’est qu’il faut qu’elles soient automatiques, et lancées périodiquement ! ce bon vieux cron est encore une fois notre ami :

vim sauvegarde.ssh

on y écrit :

#!/bin/sh
rsync -avz --delete --exclude=.*  -e "ssh -i /home/anthony/.ssh/diskstation"   /home/anthony/  diskstation:/volume1/homes/anthony/x60s

chmod +x

on peut tester :

./sauvegarde.sh

et enfin le rajouter à la crontab :

sudo vim /etc/crontab

et rajouter la ligne (pour lancer la sauvegarde tous les Dimanche à 2h00) :

# m h dom mon dow user  command
0 2    * * 7   anthony      /home/anthony/sauvegarde.sh

• Répétez cette opération pour tous les postes du réseau local.

Tout devrait bien se passer quotidiennement, à surveiller que tout est bien en place quand le disque dur de l’hôte commence à faire des bruits étranges ou avant de partir en congés…

Sauvegarde entre le NAS et un serveur distant

Le logiciel d’administration web des NAS Synology, Disk Station Manager (DSM 3.1 au moment où j’écris) dispose d’une efficace interface d’administration, qui comporte un volet « Restaurer et sauvegarder »; nous pourrions très bien faire comme précédemment pour sauvegarder le NAS vers un serveur distant, mais autant profiter de l’interface graphique et des notifications de DSM !

Pour commencer, vous devez préparer votre serveur distant à recevoir les sauvegardes, dans mon cas, nous avons un utilisateur backup, avec son répertoire personnel /home/backup

Dans ce même répertoire personnel, nous allons y créer un fichier rsyncd.conf avec le contenu suivant :

[diskstation]
comment = diskstation
path = /home/backup/diskstation
use chroot = no
read only = no
list = yes
uid = sauvegarde
gid = sauvegarde
strict modes = yes
ignore errors = no
ignore nonreadable = yes
transfer logging = no
timeout = 600
refuse options = checksum dry-run
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz

Grâce à ce fichier, lorsque votre NAS fera ses sauvegardes, il trouvera la configuration rsync.

Côté client, donc le NAS, dirigez vous vers l’interface graphique « Restaurer et sauvegarder » et créez une nouvelle sauvegarde

• choisissez sauvegarde réseau (compatible rsync)
• sélectionnez toutes les options SAUF l’avant dernière (réserver les fichiers; si vous la cochez, lorsque vous supprimerez un fichier/répertoire de la source il sera gardé sur le serveur)
• choisissez le nom de module rsync que vous avez configuré sur votre serveur distant (dans notre cas diskstation) et saisissez l’utilisateur (dans notre cas backup) ainsi que son mot de passe
• Testez la connexion, paramétrez la fréquence

Et c’est fini !

Sauvegarde entre un serveur distant et le NAS

Dans ce cas ci, vous devez au préalable avoir un moyen d’identifier sur internet votre nas, par exemple, DSM vient avec un client DynDNS.

Dans la suite du paragraphe j’assume que vous avez un nom d’hôte fixe (ou une IP fixe) pour votre NAS, et que vous pouvez y accéder par internet (pour ceci vous devrez configurer aussi les bons port forwarding [seul ssh nous intéresse dans le cas présent] depuis votre routeur ou « box »)

Pour le reste, c’est la même chose que ce que j’ai décrit au chapitre « Sauvegarde entre les ordinateurs locaux et le NAS » …

La particularité que vous pourrez rencontrer, au cas où vous sauvegarder des répertoires qui appartiennent à plusieurs utilisateurs sur le serveur distant à sauvegarder vers le NAS, est de travailler avec l’utilisateur root : c’est le seul utilisateur à pouvoir accéder à tous le système de fichiers.

Donc

• côté NAS, créez un utilisateur backup, donnez lui un répertoire personnel et l’accès via SSH;
• côté serveur distant à sauvegarder, créez un couple de clefs pour se connecter sans mot de passe au NAS, en tant que utilisateur backup, créez un script sauvegarde.sh, qui vous rentrerez dans la crontab, à lancer par root (il pourra donc travailler sur tout le système de fichiers)

procéder de cette manière vous permet de pouvoir sauvegarder tout le système de fichiers, sans jamais compromettre l’utilisateur root (pas de connexion en tant que root, ni par SSH ni par sudo, etc…)

Astuces pour la performance de votre NAS Synology

Si vous avez beaucoup de photos dans le partage « photo », vous risquez de voir un process appelé « convert » (en fait c’est imagemagick) prendre tout la mémoire vive et une bonne partie du CPU de votre Synology.

Ce process est lancé par le démon suivant :

/usr/syno/etc/rc.d/S77synomkthumbd.sh

donc il vous suffit, une fois connecté par ssh en root (même mot de passe que admin une fois que l’accès ssh est configuré) de lancer :

/usr/syno/etc/rc.d/S77synomkthumbd.sh stop

Si vous voulez définitivement arrêter ce service, éditez le avec vim et rajouter exit à la 2ème ligne (cette astuce est publiée dans le wiki de synology; attention à ne pas comme eux désactiver postgresql, toutes les applications multimédia : photo, son, video, téléchargements nécessitent cette base de donnée); attention quand vous remettrez votre NAS à jour, il faudra ré effectuer l’opération (sigh!)

Diffuser des fichiers vidéos hébergés par le NAS Synology

Le plus simple, si vous disposez d’un lecteur branché à la TV qui peut naviguer sur des partages Samba/CIFS (comme la vieille XBOX avec XBMC), c’est alors de configurer un dossier partagé (par défaut, un dossier partagé est un partage SMB/CIFS) contenant vos vidéos.

Si vous utilisez une PS3, l’approche est différente, il faut utiliser le partage médias par DLNA (car la PS3 ne peut pas monter un partage SMB/CIFS)

Si vous configurez votre NAS comme étant un serveur DLNA, il vous suffit d’avoir vos videos dans le partage video pour que la PS3 puisse les lire…enfin çà va bien pour les formats (container, codes audio et video) que la PS3 reconnaît nativement…

Pour les autres formats de video, comme par exemple container MKV encapsulant du h264, votre PS3 vous dira « format de fichier non compatible ».

Deux options se présentent à vous :

• votre PS3 utilise un firmware « non officiel »comme le CFW 3.55 kmeaw, dans ce cas vous pouvez installer le logiciel de lecture Showtime sur votre PS3, qui non seulement lit les partages DLNA, mais est capable aussi d’ouvrir à peu près tous les containeurs vidéo et recense un bon nombre de codecs..(il repose sur libav)
• Une autre approche, est de convertir à la volée vos fichiers vidéos dans un format que la PS3 peut ouvrir, le logiciel PS3 Media Player semble pouvoir être installable sur un NAS Synology, ceci dit je n’ai pas essayé…

False assumptions learned in the last 10 years (that Hadoop has been building with this in mind)

1. Machines are reliable, Hadoop separates fault tolerance logic from code logic
2. Machines deserve identities, I put data in a cluster, I don’t care which particular machine hosts the data; Hadoop can swap in and swap out machines across the cluster
3. Your analysis fits on one machine, Hadoop scales linearly with data size or analysis complexity

A typical Hadoop installation : 5 to 4000 commodity servers (8 cores, 24 GB RAM, 4 to 12 TB hard drive; 2 levels network architecture, 20 to 40 nodes per rack)

The cluster nodes are composed of m

• master nodes : 1 NameNode  (metadata) and 1 jobtracker
• slaves nodes  (1 to 4000 each) data nodes and tasktrackers

To access the file system, you would not mount it (even if you could, with fuse), you can use an API, HDFS API (in Java)

Hadoop will write on chunks of 64 MB, which will get replicated across the nodes.

Using HDFS, you will use 2 functions : map() and reduce(); they are run on the node containing the data, so no network overhead to get the info, but HDFS can interpret bytes as key; then reduce is used to aggregate the value.

Hadoop is not only map/reduce, with Hive, you can also use SQL; but there are other tools on top of Hadoop, Pig (DataFlow) or Sqoop (RDBMS compatibilty)

Who uses Hadoop : Yahoo (>82 PB, >40 k machines); FaceBook, 15 TB data/day, 1200 machines; Twitter, etc…

Mozilla uses Hadoop to analyze crash data (FF crashes, you send a report, and they get and analyze the data)

Hadoop Java brings some good tooling (along with integration tools such as Apache, Ivy, etc..) but some bad things such as JVM bugs, JNI libraries to add for non standard features (specific to the OS)

http://www.eclipsecon.org/2011/sessions/?page=sessions&id=2370

Then, Bernhard evoked Findbugs, saying that

• back in March 2004, only 4 packages for version 0.7.2, very simple
• new features added for annotations for example, few months later, still looking good
• in May 2005, a first cyclic dependency appeared, in the svn log, « temporary hack »appears
• June 2006, version 1.0.0 many packages,   new cyclic dependency,
• 2009 : many, many tangles

Can you still maintain easily this project ?

He followed his explanation with a tool called Sotoarc to show the dependency between classes and packages; he also mentioned PDE dependency visualization

To check your dependency, you can perform architectural inspection with some architectural tools.

Then Frederic explained to the audience the critical aspect of the architectural quality of an open source project: it is used by many many consumers.

He then enumerated the risks of Erosion in FOSS :

• contributors from several organizations (different processes)
• lower pressure from management
• hazardous funding

What about Eclipse project then ? Bernhard analyzed the architecture of JDT, and found out that the org.eclipse.ant.ui plugin uses JDT, and this dependency could have be avoided.

Also, duplication of code can cause problems of maintenance; he demonstrated this problem with a DialogField class in Eclipse, duplicated since 2.0

Frederic introduced to us the recommendations of the Architecture Council.

Also, some new eclipse projects, such as Orion, the web based ide, introduce new architectures.

To migrate your plugins from Eclipse 3.x to e4 , you have to go through an architectural modernization process which consists in auditing, testing, etc…

To do that, he suggests to use models to represent and manipulate artifacts of existing projects, using MoDisco; so that you can get in EMF the model of  a plugin (sources folder, plugin.xml, etc..

Once you have the model represented, you can proceed to the migration of this model to the model of e4 for example.

You need to analyze and proceed to modernization of your tools.

http://www.eclipsecon.org/2011/sessions/?page=sessions&id=2001

• Tracking memory leaks : hard to debug and leading to OutOfMemory errors
• Thread issues : deadlocks, exhausted threads, waiting for external resources, such as DB connection

To solve those problems, you can use Eclipse Memory Analyzer, which is :

• a Java Heap Analyzer
• it helps find memory leaks and reduce memory consumption
• UI and programmatic access to data in the heap dumps
• faster then competitors, such as JHat
• not that intrusive (in terms of memory overhead)
• supporting a lot of JVM (Sun, SAP, 1.4.2 and above)
• providing an automatic problem recognition

Then, on to the demo ! Dimitar,

• loaded a thread dump (very fast to startup)
• showed to us some problems found by the application, such as leak suspects
• and some views such as memory consumed by objects, the states of the threads, etc…

Back to the presentation, Vladimir explained how you can extend the tool (you can extend it to save time doing the same thing repeatedly, to share view configurations, etc…)

There are several ways to access a snapshot

• through the API : ISnapshot, Iclass, IObject, IArray
• extensions points : NameResolver (descriptions of objects) ; Query, for exploring dumps or detailed analysis, in a form of table, tree, histogram ; RequestResolver, to provide the details of what a thread is doing; Vladimir also told us about the presence of 6 other extension points

http://www.eclipsecon.org/2011/sessions/?page=sessions&id=2184

The virgo packages

• Equinox,
• then Virgo Kernel,
• then Virgo Tomcat Server or Virgo Jetty Server

The Gemini project

• blueprint : DI
• Web : support for WABs and WARs
• JPA : persistence in OSGi
• DBAccess : modular JDBC drivers
• Management : JMX management (start /stop)
• JNDI : JNDI naming in OSGi

Demo :

How to extend virgo to get your own custom server

1. Download virgo kernel and unzip it (and configure your osgi console port)
2. starts it, it looks for new applications to install; you can open an osgi console through telnet
3. add equinox.http to the ext repo, and the same for equinox servlet API bundle (2.5)
4. Configure http port , in properties, define : org.osgi.service.http.port=5555 for example
5. add the HttpService to the initial artifacts (a configuration similar to a config.ini osgi.bundles list)
6. start it, check everything is ok in the osgi console
7. You can then deploy a jar (osgi bundle) containing a servlet, configured using Spring (osgi:reference on osgi.http service); just copying it to the virgo kernel : it just works

In the works :

1. P2 integration
2. integration with Gemini projects
3. better tooling from libra project

Talking to Christopher and Glynn, I also learned that : (I’m totally a virgo n00b)

• Virgo is like having Equinox + Spring OSGi + Jetty; BUT it also provides a unified mechanism system, using logback (and so slf4j logging APIs)
• Virgo also solved problems such as Emphatic loading mechanism : take hibernate for example, a bundle using it would normally not be able to see the other bundles providing Hibernate based classes; that’s why they came up with this idea of dynamic bundle creation at runtime, that solves this visibility problem (check Glynn blog entry to know more about this problem)
• Not P2 enabled : Virgo just recently joined the Eclipse world, and since uses its own dependency and update mechanism; P2 support is high on the priority list

http://www.eclipsecon.org/2011/sessions/?page=sessions&id=2086

• What is the version of Eclipse I have to use ?
• What is the SCM URL ?
• what version ? trunk ? a branch ? if there is PSF file where is it ?
• what are the plugins I will need ?

Yoxos is about defining an Eclipse IDE profile, so that when new comers join the team, they just doucle click on a file to see an Eclipse IDE generated and ready to use with all the plugins, configurations they need to work on the project

Ian showed us a demo of Yoxos

• configuring a new profile : you can configure your update strategy : do you want your IDE to get the latest updates, the same for the plugins ?(slicing options); there is also a list of components already included to choose from (mylin, checkstyle, etc..) : the components are fetched the first time from the internet (amazon cloud + original repo), but you do not need to access the internet everytime you launch your built IDE (by the way, at startup, it can update itself); and you can share your SCM settings, project sets, workspace preferences, mylin checkstyle, findbugs configurations into the profile
• using the generated profile : from the previous step, a single file of 52 bytes contains the whole profile configuration, you launch it, and then (the first time) it is going to fetch them for you, no need to provide your dependencies and their versions, everything is resolved ; first time we launch the ide, all the projects are checked out, and all the plugins and rules are there.

http://www.eclipsecon.org/2011/sessions/?page=sessions&id=2074

http://www.slideshare.net/irbull/p2talk

Pascal and Ian created this presentation to explain what are the things developers should not do with p2.

1. Move / remove files on disk : for examples jars i the plugins folder; better than this : let p2 manage your install, so that when you remove a plugin, p2 will remove the now un-needed dependencies
2. unzip your plugins  over Eclipse; instead provide a p2 repository, that could be downloaded as a zip; by the way do not use the dropins folder; because the scan of plugins inside is so slow !
3. Replace published content, when you republish the exact same version of the same bundle (1.0.0.HEAD); p2 will not be able to distinguish it and do the updates (p2 works with bundle name + version); you have to provide version/id pairs immutable
4. Alter a released repository : people are consuming your content, so do their headless builds; a released repository SHOULD NEVER change; you can use composite repos to change content
5. Don’t categorize things; better than this, you should always categorize your repositories with a meaningful name: if you don’t like the category, during build, you can change it
6. Ignore your version ranges : for example do not specify too strict versions, if it works with 3.6.1 , it should work with 3.6.2 too ! Think about versioning your packages, bundles and features, with ranges such as : [3.6.0,3.7.0]
7. Don’t use API : be mindful of what you use, do not use the internal packages, ask on the p2-dev mailing list
8. Use the metadata generator ; instead use the publisher (p2.publisher.UpdateSitePublisher), tycho and PDE already use it.
9. Use legacy update sites : they do not include complete metadata, so P2 will download every feature to regenerate the complete metadata
10. Spell it P2, the name is p2

Great presentation ! very lively and insightful !

http://www.eclipsecon.org/2011/sessions/?page=sessions&id=2313

• Java edition : based on JDT
• Debugging : using ADB, that is running as an host on your desktop, and also on your phone or emulator; for the users, just click on the usual debug button, and the magic happens (connection between host and client adb); everything is supported, except hot code replacement.
• Logcat is the view that enables you to see all the text outputs

• Heap view : to see the the state of the memory, when profiling, results can be exported to be opened with Eclipse Memory Analyzer
• Integration testing is also made available, by deploying the integration tests package to the phone, to run it against the test app; the results will be sent back to Eclipse.
• signing your applications, using other dialogs to access your keystore.

• Text editors (XML editors) : many guis are provided : the manifest editor where you can define your your activities, versions, permissions through many tabs,
• the UI editor , where you can drag and drop widgets, all shown in a convenient drop down menu : no need anymore to go to the xml view, everything is there; moreover, you can also use the graphical editor to load a layout written in Java; there is the ability to edit nested layouts; to set the the screen size (convenient if you want to see what it will look like on a tablet compared to a phone) and also to change from portrait to landscape
• Animation editor : same thing, based on XML with a graphical editor to handle all the cases

They ended the presentation giving some links where you can find the online resources such as the source code, project site (android tooling), bugs, mailing lists, etc..

Questions :

• what about the integration of Maven, using the already existing Maven tooling for Android ? it is open source, so if they want to contribute to the Android tooling project, they could.
• Is there a CI website showing the build results ? Not a public one.

http://www.eclipsecon.org/2011/sessions/?page=sessions&id=2293

Pascal started his presentation giving the history of P2, that he described as the only provisioning system for Equinox

• 3.4/3.5 provisional api
• 3.6 (Helios) first official release
• 3.7 (Indigo) full backward release

What’s  new in 3.7 ?

• API : new features to use the consumption, simplification for RCP use case, simpler headless API for simple cases (Even simple things were complex !)
• SPI : pluggable transport (SAP needs, they target a P2 runtime less than a 1MB)
• Memory code improvement, inter process locking of local repositories

3 level of APIs :

• graphical API : SWT based, designed to be used in RCP applications (few dialogs and wizards that you can found in Eclipse update dialogs) ; use org.eclipse.equinox.p2.rcp.feature : you include that in your product and you’re done (it will include org.eclipse.equinox.p2.ui.sdk bundle) this approach was chosen because it was too complicated to the user to build his UI from « toothpicks »; tweaking the API is also made available ; and there is also an extension/ discovery UI (org.eclipse.equinox.p2.discovery, Pascal is proud of the icons work ) but API is not provided for this one
• headless operations : minimal headless P2 , via p2.core.feature enough to install/update/uninstall , you can find it from 3.7 M6 (you can still do smaller for specific use cases); headless operations : org.eclipse.equinox.p2.operations; Pascal then presented a demo plugin that uses this operation API : 4 lines of code to install a new plugin ! (compared to more that 20 before)
• core APIs and SPIs : Installable Unit, Profile, ProvisiongAgent… you do not need to know everything anymore, since the high level operations encapsulate the P2 complexity; what is not API ? the p2 folder and the repository serialized format : don’t mess with those !

1. distracted by infrastructure : the time you spend on it is a time you could have used focusing on your business
2. It’s bad mojo to pollute the POJO (Austin Powers anyone ?) : why are those developers always obsessed at getting their BundleContext ? to register services ? did you read the declarative services spec ?
3. Using Services in a risky way : Jeff gave an example with previous P2 code, using helper method to getService (find the registry, get the reference, and the service from that reference) why risky ? to use the serviceTracker safely, you need a fat amount of lines of code, there are 5 synchronized blocks, etc.. The solution ? use Declarative Services
4. Being too fine-grained : just because you can declare a lot of services, does not mean you should; think twice about your use case;
5. Putting too much in your bundle : does this class needs to be duplicated or should I create a huge bundle to contain all my utility classes ? classes have to be together for a good reason
6. Low packages cohesion : do finer grained package, to be able to evolute easily
7. Failure to refactor : don’t forget to refactor ! It’s  agile development !
8. Failure to define (good) API : with good fences and good gate it protects you from having people using this private method you clearly do not want to support, as a producer you have to make clear how to consume your API.
9. Failure to follow API : related to point 9. , follow the restrictions, if you do not, migrating to the next version will become an head ache.
10. Clinging to old ideas : it will cost you time, modularity, money if you stick on old ideas that do not take into account OSGi at all.

Stop trying to guess your start levels, think dynamic !

In a nutshell :

• Get your granularity right
• plan for refactoring
• don’t pollute your POJOs, for that,
• Use declarative services
• Manage your APIs
• Respect other people’s API

There’s a last sign you do OSGi wrong : you did not read the book OSGi and Equinox , Creating Modular Java Systems

Very interesting and entertaining presentation !

When I entered the room, I got handed over a usb stick with an archive named theinternet.zip (we’ll see what’s inside during the tutorial); the room is packed, the session is beginning in few minutes.

Presentation of Tycho;

• Jan first defined the audience what is Tycho about : a way to build Eclipse/OSGi artifacts with Maven,
• then who are the nowadays contributors : SAP, Sonatype and Intalio
• who is using it : the contributors, for sure; and EGit, P2, JBoss tools, Eclipse Memory Analyzer, JRebel, etc…
• the relation between Maven and Tycho : Tycho is solving dependencies from Manifest.MF, features.xml; and is working in a Manifest first approach compared to a pom.xml first.(but it can also be used in a pom first approach)
• the relation between P2 and Tycho : Tycho produces P2 repositories; Tycho consumes P2 repositories and its metadata
• the relation between PDE and Tycho : tycho uses the same artifacts (Manifest.mf, build.properties, feature.xml, site.xml) and is actually complementary to PDE UI; but is a competitor to PDE headless mode.
• where is tycho : open source, incubating in Eclipse foundation, code base on github

After those good words, Pascal demoed tycho

• Pascal showed some p2 projects being built with Tycho (all p2 projects contained their pom.xml, defining the name of the plugin, and the type : eclipse-plugin or eclipse-feature actually) ; he had also to add a parent pom to be able to build all at once : he cd’ed to to the p2 agregator project and a simple « mvn clean install »; later; tycho began the magic; during the compilation, Pascal talked about is PDE doing compared to tycho (for example the steps order is different) after 1min  and a half later; the crowd applauded Pascal : the p2 projects were built; pretty fast.
• Then Pascal cd’ed to another project : eclipse webtools xml; he located the parent project and launched a pom.xml : to show how easy it is for a newcomer (since he did not know this project) to build a tycho based project : just run « mvn clean install » and you’re done : perfect for getting new contributors to join (no more « it does not work on my desktop » whining anymore.)
• Adding more steps to the build; Pascal was able to demo Tycho executing the maven findbugs plugin during his build; just adding the maven plugin configuration step in the pom.xml
• Moving on, Pascal showed us the integration of the Egit Tycho build in Hudson, with findbugs report

Questions :

• is it using JDT compiler ? yes
• can i exclude plugins or features from a build ? tycho does not checkout your sources; you can use hudson for that (no maps to define what is getting downloaded)
• no possibility to build against different JDT versions; code coverage is not provided yet : Igor, Jan and Pascal answered that the API is opened and even though tycho is production ready; some aspects are not yet covered; but anybody can contribute to it to cover more particular aspects.
• Releasing the p2 repository built by Tycho to Nexus is not available now : to publish p2 repos to Nexus Pro, Sonatype sends a zip file to Nexus that explodes and exposes it; during the year, some hooks will be added to do a better integration between p2 repos built with tycho and Nexus Pro.

Coffee break : Pascal is announcing a book about tycho is coming soon (Yai!).

The tutorial

• Unzip the archive, backup your current .m2 settings folder; and create a new one with the given settings.xml present : the idea of those steps is to do everything offline, using settings that only consider local artifacts
• Start the provided nexus : /var/tmp/eclipsecon2011/nexus/bin/jsw/linux-x86-64/nexus start
• Access nexus pro (trial version) http://localhost:8081/nexus/ (admin/admin123)
• unzip a custom built eclipse : unzip eclipsecon2011.tycho.tutorial.ide-linux.gtk.x86_64.zip
• create a new rcp app; add to it the right pom.xml, and « mvn clean install « it from eclipse, using m2eclipse provided (and do not forget to use a good eclipse.ini because the permgenspace error popped in my face several times !); then we added a test (using a fragment), a feature, a repository and finally a product
• Igor then added the build to Hudson : it built fast !

Lessons learned

• stop using a site.xml; use a category.xml to define an eclipse-repository
• talking to Jan and Pascal, integration testing is not totally obvious with Tycho; I could first create a product, and run those integration tests against a built product;  instead of launching integration tests during a normal build phase; anyway from their experiences, integration testing is not that mandatory for your projects; focus more on normal unit testing first
• before defining a product, launch it a first time, and base your product definition on this launch configuration
• in a UI based  product, add org.eclipse.rcp feature; so that you will be able to build your product for multiple platforms (this whole native library stuff, swt; actually on linux x64 I did not need to add this feature to make my product work)
• Define aggregator : instead of putting all your modules in the parent pom, you can create an aggregator pom which will define your modules; so that your parent is free of modules dependencies (can be built alone); speaking with Igor, there is no « truth »about maven tycho layouts; actually m2eclipse supports nested projects in
• Tycho 0.11 will be out when it is ready; ie the product is P2 updatable without hooks
• Igor clearly recommends not using the -tycho.targetPlatfrom option; it just loads the jars in a basic way; if anything bad happens, impossible to debug
• Nexus professional will add support for hosting custom P2 repos (containing your stuff; it will allow you tycho build to publish the built p2 repo to nexus) throughout the year –> I can’t wait for this killer feature!

Great presentation & tutorial ! Kudos to Pascal, Jan, and Igor for sharing their knowledge and their patience during the tutorial !