Home

Teamgeist overview

[Sjoerd Simons’s “quick high-level overview of what teamgeist is supposed to be” as of 9 Sept 2009]

So the very high-level use-case is: you’re working on a project with a group (one could say a team) and you basically want everyone to have some indication of what other people are doing which is relevant to the project you’re working on.

You would give teamgeist some search query which would match zeitgeist events that are relevant to the project. Then the various users join a room which has a d-bus tube and the teamgeist/zeitgeist combination would forward the various events to this room so other people can keep track of them. (The UI for example could show a timeline of what people are/were working on).

Another interesting thing would be that when there are events referring to certain files it would be easy for the other users to get the updated version of the file (or it would automagically be downloaded). Or for users to add a local file to the project, such that teamgeist would update its filters to signal future events on that file to others.

Design from a GUI perspective

Creating/Joining Teams

The UI would need to have a ‘Team’ or ‘Teamgeist’ menu which would be used to access specific Teamgeist actions, such as ‘Create Team’ or ‘Join a Team’.

Creating a team would result in a window asking for the team name and identifier. The identifier would need to be valid, non-existing and unique MUC JID. The window would have a ‘Cancel’ and a ‘Create Team’ button.

Joining a team results in a similar window where the team name cannot be entered and where the ‘Create’ button would be replaced by a ‘Request to join Team’ button.

When you create/join a team, a new tab in the UI would be created to represent that team’s events.

Sharing events

What we want with teamgeist is to be able to share zeitgeist events between multiple users in an easy and intuitive way. Multiple ways of doing this should be made available to the user.

The UI (Gnome-Zeitgeist) would show your local events and you could select (none, one or many) events and choose “Share event” from either a contextual menu, a main window menu, or a buttonbar. This should open a modal window which would allow you to choose which contact or team to share the event with as well as extra options. You could also Drag&Drop a set of events into a team’s tab which would automatically share these events.

The window would have options like to permanently and automatically share similar events. a checkbox “Permanently share” would enable you to choose between thre choices (via combobox or radio buttons) :

  • Share all events specific to this document (if applicable)

  • Share all events from this application

  • Share all events from this application matching the current view

The first option would only be available if zeitgeist considers the current event to be associated with a file/document and the information on the file is available.

Gnome-zeitgeist allows you to modify your current view by asking to show only bookmarked items, or items matching a certain timeframe, or events with specific tags, or events from specific applications, or events matching a search string, etc… those settings would be taken into account in order to share events matching ‘the current view’. These settings for the gnome-zeitgeist will play the role of our ‘search criteria’ for our team members, also known as ‘Topics’.

Youness’s ramblings

The Project

What we want with teamgeist is to be able to share zeitgeist events between multiple users.

This should be a secure solution as well as an automated one, so we need to make sure that events would only be shared when they need to be shared. Multiple ways of doing this should be made available to the user.

First, we want to have a single UI to do this, so it needs to be integrated into gnome-zeitgeist instead of having a separate UI for teamgeist.

The design would be something like this:

in gnome-zeitgeist, you could right click an event and select “Share with …” which would allow you to choose which contact or team to share the event with. This would allow for manual sharing of events, which would reinforce security…

You could also have it automatically share some events, You should be able to also select a “permanently share” option. That option could allow you to share all events from the specific application, or all events relating to that single document. If you share all events from a specific application, it should allow you to specify whether you want ‘all events’ or ‘all events matching the current view’.

Gnome-zeitgeist allows you to modify your current view by asking to show only bookmarked items, or items matching a certain timeframe, or events with specific tags, or events from specific applications, or events matching a search string, etc… those settings would be taken into account in order to share events matching ‘the current view’. These settings for the gnome-zeitgeist will play the role of our ‘search criteria’ for our team members.

We should also be able to select multiple events, or to apply this ‘share all events matching the current view’ from the menu itself.

It would be interesting to also have the possibility to request a document, for example, you could see that X saved a document last night for project P, and you could request that document directly through gnome-zeitgeist, so you could just double click it and the document would open, as if it was a local document.

We have not yet discussed/decided on how this document requesting should be done, whether it would be all automatic, or whether you should request a document and the other user needs to accept it first before the file is transfered.

The implementation

The work to be done on zeitgeist would be to provide a library that could be used by gnome-zeitgeist in order to easily do all this sharing, it would be a bridge between zeitgeist and telepathy and would be used by gnome-zeitgeist directly.

The API is still to be defined, but a rough draft would be to have a single object to which we could call a simple method like ‘ShareEvent’. The destination would need to be another object that would represent a person or a team (internally represent a MUC). Another API should be ‘RequestDocument’ if you want to request a document from another user. A ‘ShareDocument’ API should also be avaiable to match the ‘Requestdocument’ one.

The object should also send signals when some events happen, mainly a signal ‘EventReceived’ and ‘DocumentRequested’.

Other signals/methods might be needed for stuff like notifying of the progress of the file transfer of a document, listing which contacts are available, their presence, their nickname and their avatar.

The API seems simple enough and should take care of most cases. gnome-zeitgeist should take care of handling all the rest, mainly the pattern matching, which event to send and which to not send, to which team/person send the event, etc.. It (gnome-zeitgeist) should also be the one to store all the config of the user, mainly which events/applications/documents are shared, what are the teams, and who is a member of which team. This will allow libteamgeist (or however the lib will be called) to stay simple and minimal and provide only the bridge between zeitgeist and telepathy.

The organs

A team or a person or a group of people should/would be represented by a MUC (Multi-User Chat) in XMPP, so whenever something needs to be shared, a MUC would be created to hold all the participants of the project.

A Dbus Tube would be created for the MUC and events would be shared through that tube.

The Dbus API is to be defined.

The events have a unique URI and we should use that URI as a unique identifier for events being shared. Also note that we do not want to mix teamgeist’s events with the local event database, so we'd need to have a database file per team.

We should also have a database for storing which events or search criterias (refered here as ‘topic’) to send for a specific team!

Event sharing will be done in two modes, first push mode for all current events. So whenever an event matches our topic, we send it to everyone in the MUC.

The second mode will be pull mode for all past events, this means that if a user was offline during a time period, when she comes back online, she will not get the old events unless she specifically requests them (when trying to view them).

Requirements

  • Bandwidth efficient

  • Secure

  • Fast

  • scalability

  • Share events matching specific criterias

  • No lost events

The uncertainties

Can we just use normal file transfers for document sharing instead of using a tube?

What happens if your ‘topic’ matches “events from this week” and two weeks later someone tries to access past events? they don’t match any topic anymore, yet they are flagged as being shared since they were shared in the past… ?

Should we use the PEP XEP for sharing events instead of dbus?

Detailed design

Database

The database will be SQLlite database containing everything we need. We’ll have one database per team and it will hold all our events.

The database will contain multiple tables, representing the teams, the topics and the events.

Local events (shared) as well as remote events would be stored in the same database, and would make no differentiation between the local and remote events. This will allow a user to request events from any other user (for catching up on missed events) and any team member would be able to provide the user with the events of the other team members, this way you can catch up to events missed even the original owner of the event is offline (as long as he shared them already with other team members that are online).

The issue with this is obviously data duplication if you have a local shared event, it will appear in both zeitgeist and teamgeist database. Should we find a better solution that doesn’t duplicate data?

Table design to be determined

Teams

The teams will also be stored in a database. Originally we would use a MUC to represent a team and the team itself would only be represented by the MUC name@domain as well as a friendly name for easy referencing.

In the future, it would be better to have some other system than MUCs for representing a team (see below for concerns), and the team database would also contain the list of team members and their respective roles.

A team will have a friendly name that can be edited, so we need to be able to send the name of the team to other team members if it ever changes. Only the team owner would be able to change its name.

Since we would use a MUC to represent a team, the team’s name could be set as the description of the MUC, this way it’s stored in the server and everyone would always get the latest team name.

Topics

Topics are what represent what a user wants to share, a single user can share multiple topics with a single team. A topic can be of multiple formats, either it can represent a single event, or represent a set of events. It can also represent an application and how it should be shared (all of the application’s events, matching a specific search criterion).

Search criteria would be represented by a different database, but it would be closely linked with the topics database. The search criteria is what represents a “view” in gnome-zeitgeist and a specific application or a specific document or a special event (representing “all events”) can be linked to a search criteria.

The topics are local, which means other members will not know what topics you are sharing and team members cannot add/remove/modify a topic from your machine, this is to avoid someone adding a new topic that matches events that you do not want to share. So topics are private.

Sharing events

Events should be shared amongst other users in a team by sending automatically any incoming event that matches any topic in the database. When such an event matches it should be added to our database (with the owner being ourselves) and should be pushed automatically to other team members depending on what the team database contains.

If an event gets shared but the user doesn’t want it shared, she can ‘unshare’ the event. By doing that, teamgeist will remove the event .

Catching up on missed events

When a user has just joined a team or becomes available, he should contact the other team members to know what he missed. He would use the last event he received from each member to notify them of what events he might have missed.

All members can tell the user what events he has missed while being offline (or not part of the team) by specifying how many events they have per timeframe, by type (X image events, Y website events, for the specific date Z). At that point, the user should have enough knowledge of what he missed, and upon request by the UI, the user can request specific events for a specific date from the other team members.

If the event he’s looking for comes from an unavailable team member, other team members who have that event can substitute for him and provide him with the missed events from the other team member. Teamgeist should then add those events but flag them as ‘unverified’. An event can only be verified by two means :
The event was received from the original author
The original author is able to validate the authenticity of that event by providing the user with a hash of his ‘unverified’ events.

Also note that when the original author becomes available, the only event we are sure to have is the last verified event, we must send the team member that event’s URI as being the last received event because we don’t know if the events we got from other members were complete (another member might have missed some events too).

Some way to automatically request and download all missed events should be made available if someone needs to be browsing the events offline and wants to catch up.

Document sharing

A document can also be shared amongst other team members in the same way the events are shared. The main differences are that :
A document is not stored in the database, but is stored in a file instead
local documents do not get copied, and only the latest version of the document (on disc) is available
document sharing only works in pull mode, if a team member does not request it, the user will not send the document to other team members
A document is viewable through its associated event and a user can request a document that is not shared (see below for this use-case)

The similarities are that :

  • The sharing of a document would also be a ‘topic’ (you can have a search criteria that says all documents are to be shared if they match, etc..)

  • The document sharing would use the same system/algorithm as the “catching up on missed events” system, and can be flag as ‘unverified’

If a document is shared (document matching a topic and is the item of event that is also shared), then any team member can request that document and it will be automatically transfered to them.

If a document is not shared, but is the item of a shared event, a team member can request that document, and the original owner must authorize the sharing of that document before it can be transfered.

In that case, a new topic must be added sharing that document but the topic would apply to the specific user requesting the document, not to the whole team (unless the user specifies that he wants to the document to be shared with everyone else in the team through the UI).

Concerns

My main concerns are for these three points, which coincidently seem to be related to each other in some way :

  • Security

  • Scalability

  • Bandwidth usage

I believe that the bandwidth needed could easily be high depending on the events being shared, Adding Pull mode (and forcing pull mode on document sharing) will help reduce the bandwidth, but real life testing would need to be done before knowing exactly how much bandwidth will be used.

However, it looks like a single event wouldn’t be very big, and the event creating rate would be slow (you can’t open/save documents and browse 100s of files/websites per minute), so the bandwidth issue would probably be only visible when trying to catch up on missed events.

The security issue is the most important.. Considering that dbus tubes in a MUC will automatically result in every D-Bus message being sent in-band in the protocol, it means that the server holder could potentially have access to all events/documents that are being shared with zeitgeist.

Adding GPG encryption would not only add a huge overhead (affects bandwidth) but would also completely kill the concept of using a MUC since GPG encryption would only work from one person to another (afaik).

The fact that a MUC is used could also potentially be harmful, if anyone would be allowed to join the MUC. That’s why we would need to design a better solution not involving MUCs, or with better MUC security (i.e. only the MUC owner can invite a user to the MUC).

Also, if you wish to share a private document with one person (your boss), transferring that file over the MUC will make all other users get it as well even if you don’t want to share it with them (as specified in the Document sharing design, a document can be shared with single team members).

Document sharing should probably use simple file transfers being done on private channels (outside the MUC) between two users. This would increase security and make use of the file transfer algorithms.

The scalibility issue is also very important.. what happens when you have a team with over 100 users, that will probably cause a huge bandwidth issue, especially if sharing a single document would result in sending it to 100 people at the same time. Also, when you want to catch up on a mising event and the original event owner is not available, and you have 10+ users with the missing events, how would we decide which ones to receive them from, if they split the task amongst themselves (to help bandwidth issues), how to divide the event set amongst them, and how to handle a non-responsive (or disconnecting) team member… it would add complication to the code.

I also have a big concern with regards to using a MUC. As I said earlier, we should enhance MUCs security by making it private, on invitation only, only the owner can invite new users, team members should be allowed to suggest a team member to the owner so he can invite them, etc… The problem is that all these settings would die forever once everyone leaves the MUC. So we would need to have a permanent MUC.

If we want to assume that every gnome3 user would be using zeitgeist with teamgeist and would have at least a few teams set up, then can we imagine how many permanent MUCs would be created on a server ? I also don’t believe that asking people to enter a name@domain for joining a team is very user-friendly, especially since they won’t know that the @domain needs to be @conference.jabber.com or whatever other MUC specific XMPP server..

There’s also the issue with the fact that MUCs are named and the name needs to be unique! I thought about having the MUC name automatically generated to be something like “teamgeist.owner–ownerdomain.teamname@conference.jabber.com” (or @teamgeist.zeitgeist-project.com) but we would still suffer from the hundreds of thousands of permanent MUCs created on the server, and if the team name is changed, the the MUC name wouldn’t match the team name anymore…

I still need to think about this stuff…

Also, what happens when the use case of gnome-zeitgeist being closed for a while, and then opened, gnome-zeitgeist gets many new events from zeitgeist-daemon but how should they be handled (can we find which are the new events since last run, we shouldn’t push them automatically, would it be cpu-intensive to get all the new events and do topic-matching on them at startup of gnome-zeitgeist)..

Another issue is about the topic matching.. should we just provide a bridge between telepathy and zeitgeist, and the topic matching will be created/stored/read internally in gnome-zeitgeist and we would only handle event sharing, or should the topic-matching be stored/handled by teamgeist ? If the topics are stored in teamgeist, shouldn’t teamgeist also take care of matching events with the topic?

Also, should we really rely on gnome-zeitgeist? Should we just have a teamgeist-daemon that could run along zeitgeist-daemon and have gnome-zeitgeist only used for setting up teams/topics, and teamgeist would communicate directly with zeitgeist-daemon to get all his events…

Overall design

The overall design is still obscure because of the above concerns…

Youness’s TODO

  • Write the module for handling the MUC and setting up the DBus Tube (0 days) DONE

  • Design a good database system and how teamgeist events should be integrated in the current zeitgeist database schema (0 days) DONE

  • Design the Dbus API for accessing teamgeist (0 days) DONE

  • Write the module that exports the dbus API (0 days) DONE

  • Write the module for accessing the database (0.5 days) (fix FindEventIds to match event template)

  • Write the module for managing teams (0 days) DONE

  • Write the module for managing topics (0 days) DONE

  • Write the module for Event sharing (0 days) DONE

  • Write the module for topic matching (0.5 days) (need to match the event with the search criterias)

  • Write the module that takes care of document sharing (0 days) DROPPED

Use cases

Team handling :

  • Create a new team

  • Name/rename a team

  • Request to join an existing team

  • Invite someone to your team

  • remove/kick someone from a team

  • Request the team owner to invite someone to the team

  • Request the team owner to kick someone from the team (should be private ?)

  • You get invited to a new team

  • You decide to leave a team

  • You get invited to a team you had already joined then left or for which you got kicked previously

Assuming you are part of a team already and it’s correctly setup. You can then setup your ‘topics’ or search criteria :

  • Share a single event

  • Share all events from an application

  • Share all events from an application matching the current search criteria

  • Share all events for the current document

  • Share a document with the team or a team member

  • Delete a topic

  • Suggest a topic to a team member

  • Suggest a topic to all team members

  • Accept a suggested topic

So you and your team share events and you want to :

  • View events from a team

  • View past events from a team

  • View your shared events

  • View all events for a specific application (from you and your team)

  • View all events for a specific document (from you and your team)

  • Request a document

  • Share a document

  • Accept sharing a document

  • view a shared document

  • Unshare/Delete an event

  • A document is requested after being deleted from disk

  • Annotate a remote event/item

  • View annotations of an event.

Usages of the feature :

  • A team of coworkers sharing events on a common project

  • A boss/project manager monitoring the work of his/her employees

  • A user who wants to have a ‘merged’ view of his events spread through multiple computers

  • Someone who wants to know who modified a shared file (ex: on network drive) over the last X days.

Some resources

irc.freenode.net, #zeitgeist GIMPNet IRC, #gnome-zeitgeist
http://www.linux-magazine.com/Online/News/Gran-Canaria-GNOME-Zeitgeist-Tracks-What-You-ve-Done
http://live.gnome.org/GnomeZeitgeist
http://live.gnome.org/Boston2008/GUIHackfest/FileManagement/
http://xmpp.org/extensions/xep-0045.html
http://gnomejournal.org/article/70/an-introduction-to-gnome-zeitgeist
http://seilo.geekyogre.com/projects/zeitgeist/
http://seilo.geekyogre.com/ — Seif’s blog
http://xmpp.org/extensions/xep-0163.html — the XEP for the PEP (Personal Eventing Protocol)