It has been a busy fall and winter for the Rational Team Concert (RTC) development teams. We shipped 1.0.1 in October, followed by 1.0.1.1 in February, and we are working on 2.0. It’s probably not uncommon to have many releases on the “go” at the same time, but I must admit that it has been a bit mind bending.

On the plus side, it has really put our tools to the test. In particular, we’ve had to port changes from one development stream to the next and closely track what changes went into which releases. I’d like to walk through how we do this.

Streams, builds, and releases

First though, let’s review the Team Concert development stream structure. We have an integration stream for the current release, in our case 2.0, that we call “Weekly Jazz Integration”, and an integration stream for the maintenance release 1.0.1.X. Each component team has their own parallel stream structure to isolate their changes. Note that the stream structure in Figure 1 was created from within the Eclipse RTC client by clicking on the “Weekly Jazz Integration” stream from the Team Artifact navigator and selecting “New > Flow Diagram…” then adding the streams of interest to the flow diagram.

Figure 1: Work Item and Source Control teams flowing changes to 2.0 and 1.0.1.1

The stream structure gives teams and developers the isolation needed and the ability to integrate using a regular rhythm. When we show this instead in a time line view of the stream hierarchy, in Figure 2 below, you can see how the levels of integration allow team isolation with continuous integration. Each stream gets a build to ensure continuous health and also that each developer has their own private builds. It’s all about running continuous builds and flowing changes between different integration levels.

Figure 2: Timeline view of the stream hierarchy (Click for full-size)

The first thing of interest to most teams is to track what changes happened between two releases. Let’s take 1.0.1 (October) and 1.0.1.1 (February) and see the changes made by the Build and Dashboard teams in 1.0.1.1. There are two key practices we use to help us track exactly what we’ve shipped.

1. We keep all the build results for milestones and releases. The build result allows easy navigation to the snapshot that was taken of the build workspace when it was built.
2. When a build is blessed, we create a release description that is shown in the project area and links the release with the build and provides values for the “Found In” field of a work item.  We bless builds by creating a work item linked to the build with a set of approvals for each component lead.
3. We promote the snapshots for blessed builds to the main integration stream so that we keep a common list of the blessed snapshots.

Finding what changed between 1.0.1.1 and 1.0.1

To find the changes between two releases, we start from the project area editor and open the “Releases” page, as shown in Figure 3. From here we can open the build that produced this release and open the snapshot. The snapshot is the Source Control artifact which references a list of component baselines. It’s like a read-only copy of a repository workspace. When a build is run, a snapshot is created automatically by the build engine.

Figure 3: The releases for Rational Team Concert

We start by opening the 1.0.1 build and then clicking the snapshot linked from the build result. Now that we hav found the 1.0.1 snapshot, from the snapshot editor we click the “Compare with snapshot” action, as shown in Figure 4.

Figure 4: Snapshot editor for 1.0.1

Now to find the 1.0.1.1 snapshot, we simply select the “Weekly Jazz Integration” stream into which we promote all the snapshots from the blessed builds. When the comparison runs we get a summary of the changes between 1.0.1 and 1.0.1.1 grouped by component. You’ll also notice that because of the process we enforced for the maintenance stream, all change sets have an associated work item which allow traceability for each change made in the release. In this case, we can confirm that most teams focused on TVT or globalization fixes to enable Team Concert to be translated into 13 languages.

Figure 5: Change Explorer showing the snapshot comparison (Click for full-size)

An alternate way of finding if a change set in another stream is to run the “Locate Change set” action from a work item or the history view. As an example, let’s try and see where the change set for work item 68677 has flowed. First, select this change set from the history view, run “Locate Change set”, you’ll get a search dialog as shown below.

Figure 6: Locate change set dialog

In this case let’s search in the flow targets for our repository workspace and see where the change set has flowed.

Figure 7: Search results from locate change sets

This confirms that the change set has flowed to both the 1.0.1.1 and 2.0 streams.

Forward porting the changes from 1.0.1.1 into 2.0

In the last section we saw how to find the changes and see if a change has flowed to several streams. Now let’s see how to actually get changes to flow from 1.0.1.1 into 2.0. The best way to forward port changes is to use the Pending Changes view to accept the changes from the maintenance streams into the current development stream. Take the example below. It shows the Pending Changes view with the latest code from the 2.0 release, and I’ve changed the flow target to accept from the “Weekly Integration 1.0.1.X” stream for the Filesystem and AgilePlanning components.

Figure 8: Merging changes in Pending Changes

As with most feature releases, a lot of files can get modified. This is highlighted above with the orange potential conflict markers which indicate that files have been modified in both 1.0.1.1 and 2.0 and merging will be required when they are accepted. Notice that the Filesystem component has ported all the change sets from 1.0.1.1 into the current release, whereas the AgilePlanning team still has a couple left to port forward.

Note: In general you don’t want to flow baselines between development streams. Since the baselines are used to mark interesting points in time along a history, when histories are merged between two streams those points are less interesting if they are mixed. In the case of backporting or forward porting changes, focus on flowing the changes and not the baselines. There is a setting in the Pending Changes view preferences which allow hiding empty baselines.

Although there are several techniques to track work in two releases (for example, creating a work item for each release in which the fix must be made), it’s still very useful to merge the change sets into each release, even if the changes are redone differently. This allows the SCM system to be used to its potential and make it easier to track which changes still need to be ported.

When changes have to be completely re-written to be forward ported, it is still good practice to accept the change set from the previous release. When conflicts occur, resolve them using the “Resolve with Mine” action to mark the changes as merged although you won’t actually take any of the file changes. The changes are marked, and the change set is now part of the component’s history. From now on you can easily track which change sets need to be merged forward.

“Why would I have to re-write a fix? Can’t I just merge it?”. In most cases when changes between your streams are small, text merging will work fine. However, there are other cases when classes are refactored, someone fixes the bug already in another way, or too many conflicting changes exist, that it’s just easier to make the a one liner fix instead of figthing with a very busy text merge editor.

Conclusion

I have shown a lot here, so let me try to wrap up with a quick summary:

1. The combination of Releases, Build Results, and Snapshots allows your teams to create traceability between official releases and the code you ship.
2. Use the compare tools to compare snapshots, streams, and baselines to find out the differences between releases or at different points of time in your release cycle.
3. Use the locate change set action to find where changes have flowed.
4. Merge your changes forward to make is easier to track the fixes that need to be forward ported.

Happy parallel development!

Jean-Michel Lemieux
Team Concert PMC
Jazz Source Control Lead

Many of you have asked if we support integrating automated Visual Studio builds with Rational Team Concert. The short answer is yes, but you’ll have to use the Eclipse Team Concert client to control and view the results.

When talking about build, it helps a lot to look at a picture. The following diagram taken from the build examples shows the different parts involved in a build. For integrations with the Team Concert client for Visual Studio you can use the continuous integration engines, integrate with the build toolkit, and store your results in the repository. Then use the Eclipse client to control and view the results.

Let’s see how this works in practice. As you would expect, as we were developing our client in Visual Studio we needed to automate our builds. We started by doing the following:

• We wrote a set of MSbuild based batch files, calling devenv and other tools.
• We tested the scripts locally, checked that the installer was created and that tests were being run. We used the Visual Studio test infrastructure, but you could easily of used NUnit instead.
• After the scripts were in place, we figured out how to have them called from a command line build engine. This basically simulates what we did in the first step, but invoked from the build engine instead of manually from the command line. The command line output is automatically attached to the build result. Of course, you have to ensure that your batch files are parameterized as you can see is the following build definition screenshots:

• At this stage we have automated builds running, but we don’t have the install being uploaded or the individual test files available from the build. To further integrate, we used the Jazz Build toolkit Ant tasks, as shown in the C++ build example. From the batch file, you can call into the Ant tasks that ship with Team Concert:

:StartActivity
java -cp %CLASSPATH% %ANT_LAUNCHER% -f “%DEST_DIR_PATH%\RTC.NET Client\jazzBuildTasks.xml” -lib %JAZZ_TOOLKIT% -DuserId=%BUILD_USER_ID% -DpasswordFile=%JBE_PASS_FILE% -DrepositoryAddress=%REPOSITORY_URL% -DbuildResultUUID=%BUILD_UUID% -DactivityLabel=%1 startActivity
GOTO :EOF

:PublishFile
java -cp %CLASSPATH% %ANT_LAUNCHER% -f “%DEST_DIR_PATH%\RTC.NET Client\jazzBuildTasks.xml” -lib %JAZZ_TOOLKIT% -DuserId=%BUILD_USER_ID% -DpasswordFile=%JBE_PASS_FILE% -DrepositoryAddress=%REPOSITORY_URL% -DbuildResultUUID=%BUILD_UUID% -Dfile=%1 -Dlabel=%2 -Dtype=%3 publishFile
GOTO :EOF

To track the progress of the builds and browse the results you’ll have to use the Team Concert Eclipse client. The following screenshot shows the build activity and downloads that have been attached to the builds.

So the good news is that you can integrate your Visual Studio based builds. Over time we will try and provide more control and visibility from within the Visual Studio IDE. Before getting started with your builds, start by reading the build documentation and the examples. Then chime in to our forums if you need more help.

Cheers,
Jean-Michel

Normally the term “polish” makes me wince a little. For a long time now I have equated that term with the pejorative notion or belief that user interface design, often specifically visual design, is something that you can stick on at the end. However, as the Jazz Project team recently finished Beta 3, its third major end-of-cycle round for Rational Team Concert, I have come solidly around to understanding the value and purpose of “polish” as a way to refine the product UI — quickly.

What has changed my thinking is the realization that so much can be done in this phase. It doesn’t trump any of the more intensive design and development work done in the regular milestones, but it does provide an opportunity to take care of a number of niggling things that were on the wish list but got supplanted by more significant development pieces.

The following lists represent a sample of the polish items completed or underway. Some are arguably not polish in that they involve more significant code changes but, big or small, all represent refinements we have deemed important and doable for the first release and are being handled in this last push before we finalize.

Some of the Eclipse-based UI changes include:

• Updating the color palette for feeds in Team Central so there are more colors to choose from, each having been tested to work well as a foreground color. This palette was previously composed of the core system colors, which are not all ideal, or aesthetically pleasing, as foreground colors. To access this new palette to color code your feeds, choose “Configure…” from any feed section’s local menu, then select a feed item from the list and choose “Edit…”. This will surface the following dialog with the updated color options:
  

  

  Here is an example of a section with feed items that have been color-coded with the new palette:

  

• Using adaptable color labels in form-based editors so they change with the OS theme. This is something the Work Item editor does already. It allows for a nice visual distinction between what is a label and what is content, as well as being accessible across all themes, including high contrast. These were updated recently in the CQ editors and will also eventually be updated in the Builds editors:
  

  

• Improving consistency between the Quick Filter bar instances in My Work view, Iteration Plan editor, Work Items explorer, and Search view. This included mainly icon updates so far, ensuring all use the same ‘Filter’ scope icon and the same ‘Close’ control. You can access the Quick Filter bar using Ctrl + F from any of these UI contexts.
  
• Refining the Getting Started message areas in the My Work view, Iteration Plan editor, and User editor. The updates here included adjustments to the message wording, what words were hyperlinks, the alignment of the ‘Information’ icon and ‘Close’ control to the top instead of centered, the use of Eclipse-style ‘Close’ control instead of the more agressive OS-based one, and refinement of the graphical rendering of the message box. The following example is from My Work view:
  

  

• Improving the presentation of, and distinction between, “Load” and “Progress” bars in Iteration Plans. Previously the coloration and labeling of these two bars was very similar — they both had green and red colors and used “Hours” as the initial label. Now the color for Load shows as blue on the left for hours booked and red on right if the team member is overbooked. The Progress bar continues to use green on the left side to show progress. When ahead the bar on the right is green, and when behind the bar on the right is red. The labels for each of the bar types is now explicitly called out with “Load” and “Progress”, respectively. In both cases, more detailed information about the nature of the content in each bar is available on hover, as shown here:
  

  

  

• Improving the input experience on estimate controls in Work Items and Iteration Plans. The polish updates included refining the look and position of the overlay image; adding more information to the tooltip text for clearer guidance on how time units can be input, e.g., showing an example with 1w, 2d, 3h, 4m, where w = week, d = day, h = hour, and m = minute, and noting the “h” is the default time unit; and adding the default “h” if the field is left without a unit having been input, as in the example below. Also, now with the automatic input of the default time unit, errors are only shown if there is an obvious input error, not if a time unit has been left out. This is a friendlier approach and means most times errors should not show on these estimate controls:
  

  

  

  

  

• Showing validation markers for required fields only after ‘Save’ in the Build Definition editor. The following screen is a snapshot of how the markers previously showed on first interaction with the editor, but now these markers only show on ‘Save’ — a pattern we try to follow in editors, dialogs, and wizards:
  

  

• Using the “Show More” action link in dialogs with long lists, such as the Baseline dialog and History view in Source Control, to allow for expanding the list incrementally on demand
  
• Reordering and sizing of the default column in the History view to aid its discoverability
  
• Adding the CQ Synchronization status to the Quick Information area of work items that are synchronized with the CQ Connectors:
  

  

• Adding or replacing a number of UI graphics (icons and wizard graphics) that were either missing or needed improvement. Of course I keep coming across more to do!

Some of the Web UI changes include:

• Improving navigation between Project Areas; the regular application pages like Dashboards, Work Items, Iteration Plans, Reports, and Source Control; and Admin. These improvements included making “Project Areas” a first-class page on the global navigation bar for ready access to all Project Areas from any point in the UI. It also included introducing a Project Areas switcher on the far right of the banner, that is scoped to “My” project areas to allow for quick access to the project areas you are part of:
  

  

  

• Adding quick navigation to “All Dashboards” from the left sidebar. We’ve long had the challenge of how to show all the Team Areas within a project with the potentially deep levels of nesting on the left sidebar. Showing them all was overwhelming and difficult to navigate through, particularly in such a constrained space. The “one-link” solution here removes the hierarchy from the sidebar, replacing it with a single link that when clicked, shows the full content on the right in the main page area. So simple, and yet it took a few tries to arrive at. We have a similar approach in Iteration Plans with the break down of plans into “My”, “Current”, and “All”, and will reuse this approach in other components over time where there are deep hierarchies:
  

  

• Introducing breadcrumbs in Dashboards and other components to aid context and traversal of paths to and from a given artifact:
  

  

• More consistent load and busy indication across the UI. Previously we had a mix of graphical animation and text used in different ways and in different locations across components. Most loading or busy indication now uses a text-based “Loading …” information string either in-line or at the page-level in the upper-right corner, as shown in this example of loading an iteration plan:
  

  

  The exception is in Dashboards where we continue to use an animation in the title bar in a couple of ways: First to provide an additional layer of feedback as the viewlet is loading, and second, when there is content already in the viewlet and we need a place to show that additional information is coming in. This example shows the first case of the loading viewlet, then that same viewlet once the content is in. The animation was recently updated from a bar-style to the circular one shown here:

  

• Moving away from dialog-based error messages in favor of contextual in-page messages for issues that occur at the page level. If you do encounter a page-level error, such as a ‘Save’ problem in the Work Item editor due to a required field not being filled in, it will now show in the editor header area, much like on the Eclipse client. These in-page messages are also used in the Jazz Team Server Setup wizard, and in the Admin area with the Web UI.
  
• Introducing an “Unsaved Work” section in the Work Items sidebar to support an unlimited number of new and modified work items. This image shows one new unsaved Defect:
  

  

• Adding Quick Search type-ahead in Work Items and Dashboards, so far. This example shows type-ahead for Connectors in the Dashboards area:
  

  

• Tweaking some layout and color issues in Work Items, Reports, and Iteration Plans, including updating the color of static titles in the Overview pages of Iteration Plans so the text doesn’t look clickable and adjusting the spacing of the Quick Information area within the Work Item editor so it is easier to read

(You might also have noticed an evolving look and feel in the Web UI, most notably in the banner and global navigation areas. I’ll tell you more about these and related changes in an upcoming blog post.)

The most recent New & Noteworthy for M6a and Beta 3 will also showcase some UI polish items mixed in with a great many more recent implementation updates. If you want to see these changes live, the Rational Team Concert Beta 3 client download is available, as is the same level of the product’s Web UI.

Once you’ve had a chance to check out some of the changes, I’d be interested in hearing what changes you find useful, what you don’t find useful, and even what you are indifferent to. And if you have any niggling UI points of your own to make, don’t hesitate to let me know about those as well! We might not be able to tend to them immediately but they are likely to get on the polish list in one of the post 1.0 cycles to come.

—
Kimberley Peter
Jazz UI Design Team

Rational Team Concert ships with a component we call Jazz Team Build. Jazz Team Build provides several build-related features for the day-to-day user and the release engineer. In addition, Jazz Team Build was designed to be used with your existing build infrastructure such as Rational Build Forge, CruiseControl, and the like. In this post, I’d like to share some of the motivations behind the design of Jazz Team Build. I’ll summarize the key features we wanted to bring to the day-to-day user, and I’ll highlight the design points that actually enable a release engineer (buildmeister) to set things up to make these features a reality for the team. Having a deeper understanding of why things were done a certain way can help you use Jazz Team Build in the most appropriate way possible, given your team’s circumstances.

If you’re completely new to Jazz Team Build, you may wish to read the getting started or tutorial documents, before reading about design rationale.

“Team First”

One of the themes of the Jazz Platform is something we call “Team First” - that is, Jazz-based tools should cater to the needs of teams engaged in the collaborative development of software. In a software development project, builds are a focal point of collaboration. We’re always discussing them with each other. As a developer, you constantly need to know if your latest changes integrated cleanly with the team, or if you’ve broken the build. To debug a customer problem, you’d like to re-create a development workspace from the same code that was used in the 1.0 build. Testers want to know what build a bug has been fixed in. For many teams, builds are the heartbeat of the project, a focal point of collaboration.

Just in case it isn’t clear by now, when we talk about builds, we’re talking about automated builds that run on dedicated build machines.

Because of the importance of regular and successful builds, we wanted to bring builds to the fingertips of the team, to make everyone aware of the different builds the team has, and to show their status and trends.

Our key goals for Jazz Team Build are:

• Make build a first-class citizen in the development lifecycle.
• Bring awareness and control of builds directly to developers, testers, and others.
• Enable traceability between builds, work items, and code.

You can see these goals alive in Team Concert today:

“Viewing recent build status trends”

“Monitor builds I’m interested in”

“Getting notified when a build finishes”

“Navigating from a work item to the build it was fixed in”

“Creating a work item from a failed build”

“Browsing the change sets included in the build”

Enabling these features for your team

The pretty pictures above are all great things to enable for the day-to-day user. However, those of you that have ever setup an automated build for a large project know that it’s a lot of work to compile, test, analyze, and package the end product. Typically, this involves many days or weeks of investment in customized scripts and tools.

So, another crucially important goal is that we wanted to enable the team keep their existing build scripts and tools. How do we bridge the gap between existing infrastructure and the first-class end-user experience described above?

The cornerstones of our solution include:

• Lightweight definitions of the building blocks for defining and driving a build
• A data model for build results
• A toolkit of commands that any build can use to inform Jazz about build activities and status
• A simple build engine

Lightweight Definitions

We don’t attempt to understand or define all the details about your build. The following lightweight elements are stored in the Jazz repository:

• Build definitions. These are definitions that can define different amounts of build detail depending on the build tools you’re using. For example, we provide an “Ant” definition template out of the box, but we also provide a “Generic” template that is basically just a set of properties that your build can use in any way it chooses.
• Build engine declarations. These declarations are merely identifiers for your concrete build engines (the actual programs running a build). We don’t attempt to connect to your build machines.

These definitions and engines enable Jazz Build frameworks and UIs to work with, and present your builds, but don’t force so many details that would constrain your choice of the concrete build engine and tools you wish to use.

Data Model for Build Results

Build results are stored in the Jazz repository. Jazz build results are composed of a flexible number of contributions (logs, downloads, links, compile and test info), allowing you to publish a result as rich, or as lightweight, as you’d like. The point here is that you may already have a significant presentation of your build results elsewhere on the web. Jazz build results don’t have to duplicate this data, they can simply link to it. Through the Jazz build result (even if it contains just an external link), traceability with work items and code is enabled.

Build Toolkit

We provide a toolkit of commands (in the form of Ant tasks) that any build tool can use to inform Jazz about build activities and status. Even the simplest build can use these commands to make the build come alive in Jazz. The team will see when the build starts, all the activities it performs, and its success or failure when it finishes.

Build Engine

We provide the Jazz Build Engine (JBE) out of the box for teams that don’t have an existing driver for their builds. JBE is a simple, yet robust, continuous integration build loop that can execute your build command based on a schedule or manual requests.

Stay tuned for more…

Hopefully, by seeing these design points, you will have a better understanding when you go to use and setup Jazz Team Build. In the last year, we’ve seen it used with various tools including Rational Build Forge, PDE Build, CruiseControl and custom internal tools. Keep an eye on Jazz.net for upcoming examples of using Jazz Team Build.

—
Ryan Manwiller, Jazz Team Build

(Note - this is part of a series of posts on Jazz-related activities @ EclipseCon 2008. See this entry for background.)

Presenter: John Wiegand (IBM Rational Chief Architect)
Talk: “Influence of Digital Communities on Software Development”
Reporter: James Branigan

John started by providing a simple classification of digital communities that exist today. He organized these communities into groups, with commerce, play, socializing, working, and learning being some examples. The rest of John’s talk was about a specific type of digital community: software development communities.

John then presented an admittedly fictional view of a development community ten years from now. This community had some very interesting features, including real-time project awareness, an open marketplace for talent, and an options market for project chances of success.

After presenting this possible future software development community, John delved into a description of a growth path that could lead to these development communities of the future. The first part of this path is well-worn today, however the second part is more speculative and experimental.

He started by describing the dynamics of a team of one. With this as his base case, he expanded this to a small co-located team. From here, the team expanded to a group of distributed teams. The next step was describing what is needed to build a community around a group of projects which are composed of many teams. Eclipse itself was used as a example case of this arrangement.

The subsequent steps start with some experiments that IBM is currently running in the area of open commercial software development for the Jazz Platform and Rational Team Concert. The idea behind open commercial software development is to expose the development cycle to customers in a transparent way, so that they understand what is planned, what work items exist, and have visibility to other development artifacts. John presented the following framework for building future development communities.

How we succeed at getting to vision:

Transparency - Everything must be visible

Enable correlation of

teams way of working – Each team with its own process for working
team’s measurements – combination of metrics, not one single metric
team’s actual success – NOT claimed success

Remove barriers to collaboration

shared vocabulary – avoids the tower of babel problem
shared measurements – make sure a team cares about the same things
shared resources – avoid duplication of effort
shared practices – ensures consistency

Iterative Improvement – we won’t get there in one big step, it will be many small steps

The talk went a little over, but several people stayed afterwards to discuss the ideas for making the next steps towards the software development community of the future.

(Note - this is part of a series of posts on Jazz-related activities @ EclipseCon 2008. See this entry for background.)

Presenter: Kimberley Peter (Jazz UI Design Lead)
Talk: “Building On Eclipse In Interesting Ways - While Still Respecting Its Look and Feel”
Reporter: Patrick Streule

Kim gave an overview of four conceptual themes that we’ve applied in many places of the Rational Team Concert user interface. She outlined the theoretical background of the themes and gave practical examples of how these concepts are being used in both Eclipse and Team Concert.

Theme 1: Condensed information spaces: How do we present rich information in a limited space? How do we draw the user’s attention to the most important information first? Among other examples, Kim showed how the Jazz Work Item editor uses user initials and color to convey comprehensive information about subscribers and their online presence in a very small area.

Theme 2: Contextual linking: How do we bring related artifacts and function into the context? One example was how Team Concert uses ‘Getting started’ links to guide the user to initial or required actions while at the same time avoiding empty views.

Theme 3: Pervasive progressive disclosure: How do we go from overview to detail, allowing the user to see more information when needed? As an example, Kim showed how Team Concert’s notification slide-outs can make you aware of a build failure and then allow you, if you wish, to proceed seamlessly to the build result editor in order to get the full story.

Theme 4: Adaptable color: Kim’s favorite topic of the four, and also the most complex one: How do we render interface elements elegantly and at the same time stay adaptable and accessible? Kim described a technique of how to blend colors adaptively by detecting the RGB range of a predefined theme color.

Several members of the audience made the point that while they see the value of these visual design techniques, their teams don’t have trained visual designers on staff. These audience asked where they could get visual design guidance in the absence of trained team members. Kim suggested that they look for attractive themes and UI patterns in Eclipse and other tools and products. She also suggested that both the Eclipse and Rational Team Concert New and Noteworthy documents tend to draw attention to new and improved visual designs.

(Note - this is part of a series of posts on Jazz-related activities @ EclipseCon 2008. See this entry for background.)

Presenter: Kai-Uwe Maetzel (Jazz Process Lead)
Talk: “What if your tools knew your team?”
Reporter: Jim des Rivières

Kai set the stage by telling 4 stories from everyday life that each tell us something about collaboration. Underlying any collaboration is a set of rules and patterns, with certain key characteristics. They are generally goal-specific. They may be agreed upon up front, or emerge over time. They evolve over time based on feedback. They incorporate situational sub-patterns and rules that bring in time-sensitive dimension. And they range from generic to actor-specific. The set of rules and patterns underlying any collaboration serve to make the collaborators predictable to each other, allowing the parties to effectively coordinate their actions towards a common goal.

Jazz is taking the next step in making software development tools smarter and more helpful. By making explicit the collaborative rules and patterns that teams follow, the tools are in a position to be more helpful. Analogy: the refactoring support in JDT leads to productivity gains by taking away some of the manual code editing burden, thereby freeing the programmer to worry about more important things. The collaboration support in Jazz does the same sort of thing, helping the programmer to collaborate effectively with their team while reducing the manual effort of following the teams rules and patterns.

In Jazz, the support for collaboration rules and patterns are collectively referred to as “process”. Process support is a fundamental part of the Jazz Platform kernel. It provides full-fledged process enactment, where the system is able not only to guide and advise, but also to enforce defined rules when required. The process is “live”; changing the process immediately changes the system’s behavior. And while the Jazz team happens to subscribe to many agile practices, the Jazz Platform itself is process-neutral, allowing the system to honor the diversity of other development teams and their processes.

Kai then gave a demo of Jazz and showed the ways in which the system’s process-awareness subtly permeates many aspects of how things get presented at the UI, and informs how the system reacts to the user’s requests. He concluded by inviting people to bring their questions to the 8pm Jazz Live session, where he’d be continuing the demo.

(Note - this is part of a series of posts on Jazz-related activities @ EclipseCon 2008. See this entry for background.)

Presenter: Jean-Michel Lemieux (Jazz Source Control Lead)
Talk: “Source Control in Jazz”
Reporter: Jim des Rivières

Jazz SCM is a modern source control system based on change-sets. Team members make changes to files; these changes are collected into change-sets; the change-sets are shuffled around and combined to form change histories. In effect, change-sets are the currency in which the team members trade. Pass me the change-set with your fix for bug 12345? Let me deliver my change-sets to the stream. What change-sets made it into the latest build? Jazz exposes the essential functionality of a source control system, without the traditional SCM jargon, such as “branches”, that tends to scare users. It was designed to make parallel development really simple. It has a notion of component, each with its own change history. Components allow scaling up to teams of teams. Jazz also plays well with others. The Subversion co-existence story allows Subversion to be used with Jazz work items and builds and supports bi-directional linking. And the ClearCase connector component allows synchronization between ClearCase and Jazz SCM.

Jean-Michel, or J-M for short, then demoed the Rational Team Concert client plug-ins installed into an Eclipse Platform 3.4 and CDT (just to make the point that Jazz is language-neutral and independent of JDT). He noted that, unlike past years, everything he’d be demoing can now be downloaded from Jazz.net. He imported a pre-created CVS change-set archive file into a Jazz repository workspace. He then showed that each of the change-sets in the history could be inspected, with the delta shown in the Eclipse compare editor. He then loaded the repository workspace into his Eclipse workspace, and pointed out that the team had made every effort to make that operation fast. He showed how to discard change-sets from the repository workspace, and to cherry pick change-sets from other baselines and streams. The Eclipse workspace is automatically kept in sync with the repository workspace.

Team members deliver change-sets to a stream to share them with the team. Conversely, they accept change-sets from the stream to catch up with the rest of the team. When a conflict arises, the Pending Changes view shows the change-sets as unresolved. The conflict information is stored persistently on the server. Team members can inspect and resolve the conflicts; the UI for doing this is with a non-modal.

J-M concluded by promising to continue the demo live in the 8pm Jazz Live session.

(Note - this is part of a series of posts on Jazz-related activities @ EclipseCon 2008. See this entry for background.)

Presenter: Erich Gamma (Jazz Technical Lead)
Talk: “Developing Software like a Band Plays Jazz”
Reporter: Darin Swanson

This afternoon at EclipseCon 2008, Erich gave an overview of how Rational Team Concert, built on top of Jazz, addresses common pain points of software development. These are my highlights from Erich’s presentation.

We started with a history lesson of Jazz from the presentations at previous EclipseCons:

• 2006 - Fixing a bug
• 2007 - Development iteration with Jazz
• 2008 - Jazz life

Erich reviewed Eclipse and Jazz, their relationships, and that you can use all of your favorite plug-ins from the Eclipse world with Jazz.

Then Erich defined Jazz and Rational Team Concert to avoid any confusion:

• Jazz is a technology platform
• Rational Team Concert is the first Jazz-based product; it includes Jazz Source Control, Jazz Team Build, and Jazz Work Items

Erich drew an analogy that the Jazz Platform allows an individual to work in the context of a software team like a band member plays an instrument as part of a band; an individual making individual contributions but at the same time acting as part of a broader collaboration.

Each team needs to find the correct “balance of collaboration and solo expertise” and each member needs to have awareness of what others are doing.

Jazz technology provides tooling to reduce friction. By friction we mean “effort that doesn’t directly help in the creation of quality software”. An example of friction is a project manager polling each member of a team for a status report, wasting the project manager’s time and each team member’s time. Jazz technology removes this friction by automatically creating a report based on other artifacts like work items and change-sets and allows the project manager to see this report without bothering other team members.

Jazz technology helps people work together to be more effective through:

• Deep integration of various tools (source control, work items, build, planning, etc.)
• A “Team First” perspective when designing tools
• Promoting collaboration and transparency
• A way to make tools less visible and allow seamless completion of a task from start to finish

The structure of Rational Team Concert builds the collaborative development environment through a single repository, integrated version control, and a build system. Team Concert also has the ability to play with others with varying degrees of interop:

• Import: one-way migration of data from another tool into the Jazz equivalent (e.g. migrating Bugzilla bugs into Jazz Work Items)
• Connectors: two-way data sync, one team working completely in Jazz-based tools but collaborating with teams using other tools (e.g. Jazz SCM to ClearCase)
• Bridge: light-weight linking between a Jazz-based artifact and an artifact from a non-Jazz tool (e.g. connecting a Jazz work item to a Subversion revision)

Within Jazz, teams are at the center of devlopment and each team is allowed to be different. A project configures a basic set of rules and artifacts (e.g. bugs, tasks, stories) and teams within the project are able to customize the process to suit the team’s style of working. We enable this level of configuration via process templates and process customization.

Erich then demoed joining a team within Rational Team Concert. You add a user to a team and Team Concert asks you if you want to send the new team member an invitation, which contains all of the information they need to get started including automated configuration of a Rational Team Concert client and a set of “getting started work items” for the new team member.

Erich then continued on to demo recreating a workspace from a build to fix a specific problem. He created a change set, attempted to deliver and then used the Team Advisor view to explore the team’s process towards successful completion of his task.

Jazz is built to support the organic scaling of work from a single user, to a team, to teams of teams.

• For the user we have repository workspaces, private builds and allow customization to focus in on the key events for that user…”My events”
• For the team we have team streams with sharing of change sets, continuous builds to establish the pulse of the team and the generation of team events for coordination and notification.
• Teams of teams work with integration stabilization streams sharing baselines to produce consumable integration builds.

The Jazz platform infrastructure supports both an open source and enterprise stack:

• Open source - Tomcat, Derby, Jabber
• Enterprise - WAS, DB2/Oracle, Sametime

Erich gave some quick highlights of the components that make up Jazz leading to finish with a brief New and Noteworthy since EclipseCon 2007.

I could reiterate the text here on the changes since last year and what we think is cool within Jazz, but the most important change since last year is that you can determine all this for yourself.

As part of the Jazz.net community you can download, try out, and provide feedback on Jazz. Go give it a try!

As Jean-Michel mentioned a few weeks ago, we have a strong Jazz team presence at EclipseCon 2008 in Santa Clara, California this week. Since not all Jazz.net community members can be at EclipseCon, but may be interested in Jazz activities there, we’re going to post reports on several of the talks that Jazz team members have given.

So watch this space over the next couple of days for Jazz @ EclipseCon 2008 reports!

Update: I’ll post links to the reports below as we get them:

• Erich Gamma on “Developing Software like a Band Plays Jazz”
• Jean-Michel Lemieux on “Source Control in Jazz”
• Kai-Uwe Maetzel on “What if your tools knew your team?”
• Kim Peter on “Building On Eclipse In Interesting Ways - While Still Respecting Its Look and Feel”

—
Bill Higgins
Jazz.net and Jazz Web UI Teams

Chris Daly’s entry “Some Notes on Rhythm” provides a good high-level overview of how the Jazz Project establishes a “rhythm” to make consistent and predictable progress over the course of project milestones and release cycles. In this entry I’d like to drill-down to show how we use our tools to keep a rhythm for day-to-day development. Specifically, I’ll talk about the process that the Jazz Repository team uses to manage both our team streams and our deliveries to the project’s integration streams.

But first, a word about the goals of the Jazz Platform’s Repository component. The Repository component sits below all the other components in the Jazz stack (see diagram below), and provides fundamental capabilities (like the ability to save and fetch items from the database) that upstream components build upon. The biggest problem we have hit is trying to balance the stabilization of code we have previously delivered while still making forward progress on new features. The other teams are trying to do the same thing, and usually want to see a preview of the new features so that they can plan and adjust for things coming.

We’ve addressed this problem using Rational Team Concert’s SCM capabilities, which provide support for multiple streams and component baselines. At the Jazz project level the main stream is the Weekly Integration stream, which tracks the most stable version of every component for that week, and the secondary stream is Continuous Integration, which tracks each team’s latest (but pretty stable) code. Each team’s baselines, which capture the the stable state of a component (similar to a CVS tag), flow from the Continuous Integration stream to the Weekly stream each Monday and code is stabilized throughout the week until each team is happy with the build. During the week bug fixes flow into the Weekly Integration stream, and new features flow into the Continuous Integration stream for early testing and adoption from other teams. The balance of the two different goals is maintained by having the Weekly stream be geared towards stabilization and the Continuous stream geared towards new features and ongoing development and non-critical bug fixes.

When I’ve completed work on a new feature for the Repository component, I deliver my change-sets to the Repository development stream and eventually those change-sets flow into the Continuous stream for other teams to see and use. Each team decides how often they want to deliver the code from their development stream to the Continuous stream, and their choice could depend on how stable they think their development stream is, or how much people want to look at their newest features. For the Repository component, we have tried to push our code to Continuous almost daily so teams can leverage new features quickly, and so we can get good test coverage on low-level changes even before the Weekly stream is updated.

If a bug is found during the testing of the weekly builds, then the situation gets a little bit trickier. I need to get the version of our component that was used in the weekly build so that I can create a fix that doesn’t depend on any new changes, and eventually I need to get my change into the Weekly stream so that a new build can be started. On the Repository team we have solved this problem by adding a new stream into the mix, the Repository Patch stream, which tracks our latest contribution to the Weekly stream and provides a place to gather bug fixes for testing before a new baseline is delivered to the Weekly stream. So, to fix the bug I simply grab the latest code in the Repository Patch stream, fix the bug and deliver my fix back to the Repository Patch stream. The fixes that get put in the patch stream will flow back into the Weekly stream when they have been tested and all the pending fixes are ready.

The last part of the problem is: how do all of the changes magically flow from stream to stream? If I deliver a bugfix to the Patch stream, how do I know that I didn’t break something before it gets put into the Weekly stream, or if I deliver something to the Repository stream, how does it eventually make it’s way to the Continuous stream? Our solution to that is to have a release engineer role which we rotate among team members. Whoever is currently the release engineer manages moving our code from stream to stream on schedule and makes sure that our new deliveries don’t introduce any problems. Since everybody takes turns wearing the release engineer hat, you only have to focus on the process every couple of months, and we benefit from the fact that everybody becomes familiar with the process and we don’t become overly-dependent on a single rel-eng expert. In addition to learning the process, the release engineer becomes more familiar with Jazz SCM beyond simple ‘accept’ and ‘deliver’, such as managing baselines and working across multiple streams.

To summarize, and for visual learners, here is a diagram that shows the details or our daily flow of changes between each of the streams from the Repository perspective; the dotted-lines represent change-set flows while the solid lines represent baseline flows.

—
Matt Lavin
Jazz Repository Team

It’s funny to say, but EclipseCon is much more than just about Eclipse the IDE. It’s about tools, platforms, runtimes, and grassroots hacking. Take Jazz as an example, where I would say the most important part of Eclipse we use is the OSGi runtime on the server. Whereas on the client side our design has to allow for different front ends, like web browsers, IDEs such as Eclipse and Visual Studio, or the command line.

The Jazz team will have a big presence at EclipseCon this year. On Tuesday several Jazz team members will give Jazz-related talks including a talk I’ll be giving on Source Control in Jazz. On Tuesday night, make sure to join us for the Jazz Live! event where several senior Jazz development team members will be available to demo and chat about their particular areas of Jazz.

Along with my Jazz talk, I’ll also be giving a long talk called “From the RCP Book to Reality”. Another thing I’m hoping to do, if time permits, I’d really like to spend some time at the hackathon with someone interested in writing a PDE feature-based launcher for Eclipse and OSGi bundles. I’m tired of having long bundle lists and would like an easier way to develop Jazz which has hundreds of server, client, and common plug-ins that are already nicely separated into features.

I look forward to seeing you in California next month!

—
Jean-Michel Lemieux
Jazz Source Control Team

Recently, the Jazz Jumpstart team published a Rational Team Concert User Workshop on Jazz.net. Our small team of Paul, Steve, Philippe, and I work with Team Concert beta customers who are learning how to do serious work with this new platform. We developed the workshop to help these customers get up to speed on the technology.

Creating the Team Concert User Workshop was an interesting experience because it demonstrated to us that the collaborative nature of Team Concert is fundamentally different from past products, which focused on making an individual more productive.

Prior to working on the user workshop for Team Concert, I developed and delivered Eclipse user workshops for customers, other companies, universities, and the public. The Eclipse user workshops followed a typical workshop format of a series of instructor lectures and participant exercises. Our first attempt at a Team Concert workshop used this format. We lectured on the features of a particular component like Jazz Work Items or Jazz Source Control and the participants, using their own local Jazz Repository, would perform the related exercise. These initial Team Concert workshops failed miserably (fortunately, it was a trial run with a friendly audience). The issue was that the participants worked in isolation and did not experience Team Concert as a team collaboration platform. They came away knowing more about the functional “speeds and feeds” of Team Concert but nothing about how it helps teams collaborate better. Back to the drawing board!

We needed to move away from the functional approach and instead teach collaborative software development using Team Concert as the enabler. We had a team: our workshop participants. We needed a reasonable application development scenario that the instructor and the participants could work on together as a team. It was reasonable to expect that the participants would not know each other and their development skills might vary significantly, yet we wanted them to be a team for a day and complete the delivery of a full application milestone! The application needed to be dead simple so we could focus on collaborative development and not programming skills. Yet it had to be rich enough to experience Team Concert’s collaboration features as you might with a real software application by a real development organization. And lastly, it had to be open-ended to support any size workshop.

From these needs, the “Squawk” application was born. Jumpstart colleague Paul VanderLei created it as the simplest collaborative application you can imagine - it consists of a set of “squawker” classes that output or “squawk” some text to the console. For example, a lion squawks with a roar and Fred squawks with “Wassap?” Each participant authors his or her own squawkers to say whatever the participant wants them so say. A few lines of code and you are done. The application UI aggregates all the squawkers into a squawk party and the output is a cacophony of unusual noise. One creative participant even managed to squawk with a real audio recording!

To make this feel real, we defined a full release cycle with two development milestones and appropriate development plans. We also wanted the workshop participants to experience the challenges of component-based development and to learn how Team Concert helps you gracefully manage code dependencies. The Squawk application team structure organizes teams by component and the participants are members of two teams, Core and Documentation. Two builds were defined: one for the Core component and an integration build of all the components. In one (very full) day the participants experience a complete development milestone in which they plan, author, test, and document their squawkers. As the milestone nears completion they enter an end game phase and experience a stricter process enforced using the Team Process component.

Here is a screen shot of the team organization with some predefined users. The teams are organized around the application components.

Here is an image of the Squawker project as seen in the Team Artifacts view. It has working builds, plans, and a set of code streams that map closely to the team structure. Note that the User Interface team has a stream with a dependency on the Core team’s work. In the workshop you learn how to properly deliver changes to APIs through a managed adoption process so that upstream components aren’t left broken which might upset your teammates.

What’s fun is to see the collaboration in action. The workshop participants are all connected to a Jabber-based chat session allowing the participants and instructor to easily chat or share links to Team Concert development artifacts. You can see a plan blossom before your eyes as participants create work items for their squawkers. As code gets delivered you can watch incoming work arriving for acceptance to your development workspace. Everyone immediately sees the results of builds as the notifications arrive.

This approach has worked out very well. The participants immediately become fully engaged in the workshop because they are working together (there is no chance to sleep in the back of the room!). But most importantly, they use Team Concert in the way it was intended and experience best practices that the Jazz development team has learned using Team Concert as their development platform.

This was validated again recently when my Jumpstart colleague Steve Wasleski did a workshop at a Java user’s group in Stuttgart, Germany where about 18 folks from a variety of organizations came together for a day and completed a milestone of application development the Jazz way. The workshop participant feedback was very positive.

Now that we have some proof points like Stuttgart and elsewhere we have improved the material and the Squawk database and published the workshop on Jazz.net for you or your teams to experience. Our hope is the material is rich enough for self-study or to help teams learn together. There is undoubtedly room for improvement and we look forward to your feedback on the forums or contact us directly by creating a work item with category “Workshop”.

—
Jim D’Anjou
Jazz Jumpstart Team

Something I’ve learned in working as a user interface (UI) designer on an agile development team, as with the Jazz Project, is that you don’t so much prescribe design as you iterate on it.

Ideally you first consider what the viable design options are, and having chosen the best known path, you start with a sound design and blueprint for construction before time is invested in the code, but with concurrent development across multiple teams, it is often not possible to do this before implementation gets underway. It is a necessity of scale then that iterative design cycles get initiated from two directions: either they start with the design or they start with the implementation.

As a result, a number of different and sometimes conflicting UI patterns can emerge. This has its challenges — particularly if you want to achieve a certain level of consistency and quality across the UI — but as time goes by and you can take regular steps back to look at the system as a whole, you can see what’s working in and of itself, what things that have similarity converge nicely, and what areas diverge unnecessarily. This has been the case in the design and development of Rational Team Concert. At a certain point in the maturity of the product development cycle, exemplary interaction and presentation patterns have come to the fore, and become the ones we point to for inspiration or copy when the use case begs the same approach. Others emerge and we fine-tune them as we go. This is one technique we use to refine the user interface.

If you are interested in seeing some of these patterns, one way of course is to play with Rational Team Concert directly and discover them in a free form way, such as Edit-on-Demand in the Web client and Progressive Disclosure patterns in the both the Web and Eclipse clients.

Example 1. Edit-on-Demand in the Web client showing the dynamic presentation of the “Remove” action on the Work Items Links page. This function is available only in parts of the Web UI but will be available more pervasively soon.

Example 2. Progressive Disclosure on the Web client showing a hover with live links to allow drilling down within a results page. We use hovers in the Eclipse client for similar types of drill down.

The other route is to look at what we have captured so far in our UI checklists: The User Interface Checklist for the Jazz Eclipse Client is published under the Learn section on Jazz.net, and the User Interface Checklist & Style Guide for the Jazz Web Client is in the works on the Jazz.net wiki. Neither of these is complete, but they should give you a taste of some of the underlying principles and patterns you will encounter as you work with the product.

I also plan to use the Jazz Team Blog to feature some of the patterns and related UI elements at various stages, so watch for those posts and feel free to add comments on what you see.

This leads me to another aspect of refining the UI: feedback. On the Jazz Project team, we provide feedback to one another along the way through Work Items, reviews, and informal discussion. We also rely on, and greatly appreciate, feedback from the many of you who are registered with Jazz.net and using Rational Team Concert. Your hands-on experience with the product can help us make incremental improvements to the user experience. If you have questions or comments related to the UI, the Jazz Forum is one way to have a voice and engage in discussion. Another is by filing Work Items. And finally, I look forward to getting your feedback here on the various UI-themed posts to come.

—
Kimberley Peter
Jazz UI Design Team

I joined the Jazz project while it was still in its very early days, roughly two and a half years ago. At that time, the Raleigh team was just finishing up a short-lived experiment of trying to build an extensible platform using J2EE. This experiment demonstrated that the J2EE classloader hierarchy was not the right technology to use when building an extensible server platform. The isolation of the classloaders for each of the WAR files fights against the desire for extensibility. The primary alternative was to package all components in the same WAR file. This mega-WAR option was a deployment nightmare, not to mention exposing the implementation of components to one another. What was really needed was a viable server-side component story.

Fortunately, around this time the Equinox project was experimenting with the start of what would become known as the OSGi Bridge Servlet. This allowed an OSGi runtime to execute within a servlet container. The bridge servlet became a fundamental building block to the Jazz server design. The details of the development of the bridge servlet work and the development of the subsequent services architecture will make great fodder for a future post. For the purpose of this little history lesson, the important part of the bridge servlet is that it allowed us to solve our extensibility problem by providing the bundle and extension point based programming model of OSGi and Eclipse rather than relying exclusively on the J2EE programming model. Once this model was in place, we moved quickly into self-hosting on the components. We started self-hosting on the Team Build System, then on Work Items, and later on Source Control and the other components.

Once we started self-hosting, a couple of interesting data points started to show up in our server stats. The most interesting had to do with the cacheability of collaborative data. For performance reasons, the Jazz server has two write-through internal caches that reside in-memory. One tracks the current state of an item while the other caches a complete state of an item. We started observing current cache hit rates around 90% and state cache hit rates around 85%. This is substantially higher than can be expected from standard web data, which according to literature hover around 50%. This fact got us thinking that if we could find a way to expose our data using GET operations and ETag and LastModified headers, that we could leverage proxy servers to achieve tremendous performance gains. We made the observation that since the developers at a given location tended to work on the same component(s), that they could achieve even higher cache hit rates. This observation got us interested in exploring a move to a RESTful server architecture. However, cache hit rates weren’t the sole thing that pushed us towards a more RESTful server design.

The server interface for the Jazz Team Server that ships with the Rational Team Concert Express Betas is primarily a remote procedure call (RPC), marshaled object system. One of the key problems with this type of system is that it hides the user data in custom formats behind programmer-defined services. Hiding user data is not a desirable feature for an architecture on which you want to build a new platform for all Application Lifecycle Management tools. Rational started an effort to create a set of architectural principles as a guiding philosophy that recognizes the value of dealing with user data in an open way. The principles were validated against several experimental servers, one written in Python. An embodiment of the principles envisions a RESTful platform built with Atom and Http, and published resource formats that do not lock-in user data. Combined with our caching statistics, the architecture principles provided a strong impetus for the Jazz Platform development team to investigate building a RESTful platform in anger.

As with any design change, our experiments with REST started small. The Team Concert Betas contains code from these first experiments with REST. There are several interesting pieces that take baby steps from the existing implementation towards a RESTful approach. However, it is very difficult to move piecemeal from a RPC-based system with complex marshaling of objects to a resource-centric approach. This realization led to the establishment of the Jazz REST Services (JRS) incubator. The goal of this incubator is to develop a RESTful platform for the Jazz Team Server that embodies the architecture principles. I’ll be posting more about JRS in the near future. If you can’t wait and want to learn more now, you can check out my teammate Simon Johnston’s developerWorks blog. Simon has done a good job of providing a nice overview of JRS and its capabilities.

—
James Branigan
Jazz REST Services Team