Recipe: Migrating from Novell Groupwise to Microsoft Office 365 Part 2 – Understanding the Mailscape

After getting a clear picture of why users wanted to migrate from GroupWise to Outlook, we knew we need to look closely at exactly what we were migrating. We had very raw numbers for GroupWise – we knew only that there were approximately 5000 mailboxes totally over 30 terabytes of storage. This worked out to roughly 6GB per mailbox, which seemed awfully high.

It also meant that we had enough mailbox data that migration to Office 365 would not happen overnight – it would take days. So a third-party specialist was brought in to do analysis on exactly what was happening in Groupwise. We need to know exactly what the mailbox landscape was. What we found was shocking:

  • Over 30 terabytes of mail total
  • 404 mailboxes larger than 15 GB
  • 101 mailboxes larger than 100 GB
  • 13 mailboxes larger than 200 GB
  • 3 mailboxes larger than 300 GB (and still growing!)
  • Many mailboxes had huge numbers of subfolders (the record was 6838 subfolders)
  • Almost all mailboxes were configured with shared folders between multiple users
  • 563 mailboxes were orphaned – no one had access to them
  • 1609 mailboxes had been converted to resources, meaning they didn’t receive mail, didn’t consume a license, they only provided archival access to mail
  • 60 mail servers across 72 physical offices, geographically distributed
  • Each server responsible for its own backup, no synchronization
  • No archiving mechanism beyond backups

The analysis also showed that because the mailboxes were so large, if the biggest, busiest mail server in the group had failed, it would take almost a week to restore the data.

Clearly people were using GroupWise for far more than just email, contacts, scheduling appointments and keeping track of tasks. There was only one way to find out what people were using there system for beyond the obvious and that is by talking to them. With a user base of close to 5000 talking to everyone was impractical, so we started with users that owned the sixteen biggest mailboxes (200 GB+ and 300 GB+). After that we spoke to a random number of people that had an inbox of more than 100 GB. Last but not least we spoke to a random number of people that were having an inbox bigger than 15 GB. It turned out that the mailbox sizes correlated with the way they used the system.

The users with the biggest mailboxes usually had no shared folders or other users accessing the box. They were using the email system as a document management system (DMS). They went as far as emailing documents to them to store it in their DMS. Every mailbox bigger than 200 GB was actually being used as a DMS – the size purely related to how long the mailbox had been open and who made the largest documents.

Mailboxes bigger than a 100 GB had a common but different characteristic. These were typically a shared mailbox for an entire department. The rational for this was that everyone should have access to all mail received by anyone in the department excluding actual private mail. The department head selected a mailbox and setup proxy rights for everyone in the department to share the “departmental emails”. These mails were mostly stored in “topic folders”, and sometimes these folders had subfolders organized to store mail from particular periods, such as “Jan 2011”.

The 15 GB+ mailboxes were mostly in use by individuals and small groups who used shared folders a lot. It turned out that most users in this group were sales people and they used the system in a slightly different way as the other groups with the largest mailboxes. The overall discovery on the mailboxes smaller than 15 GB but still substantial size was that people just didn’t cleanup their mailbox and their mailbox size was influenced by the number of years they worked at the company – the longer you worked there, the bigger your mailbox.

Further investigation showed that a number of people were violating the email compliancy rules set by the company. Forwarding rules were used to enable users to work on their emails from home, which was strictly forbidden by the company.

The scope of the issues discovered within GroupWise made it clear that a quick migration was out of the question – we needed the change the way email was used by the employees of the company before we could migrate. Ultimately this broke the migration project into a number of other smaller projects.

For the sales staff, implementing a true Customer Relationship Management (CRM) system would move a lot of CRM-related data out of the email system. There are numerous products in this space, SalesForce arguably being the most famous. Ultimately the company went with Microsoft CRM Dynamics because it integrated well with Outlook and Exchange.

But the far larger issue was a proper document management system. The users had started storing documents in their mailboxes not so much by choice, but because there was no other way to manage all of their documents. The fact that it had swollen their mailboxes to massive sizes had no impact on the users as long as the servers didn’t crash. And if-and-when they did crash, the users certainly didn’t see it as the fault of their giant mailboxes – it was just the IT department failing to do its job.

A separate project was initiated to set up Sharepoint 2010 as the document management solution for the organization. Initially it was developed in-house to “lighten the load” on the email system, removing emails stuff full of attachments and replacing them with emails to links to Sharepoint documents. Ultimately the Sharepoint implementation will be migrated to Office 365 as well, since it is part of the offering.

Finally, there was the issue of email governance – up until now the company really had not enforced many rules on how email was used, such as limiting the size of email boxes and requiring archiving for size management and legal requirements. These sorts of rules would have to be applied during migration, utilizing the features in Exchange 2010/Office 365 to reduce the size of mailboxes to less than 5 GB. While Office 365 supports 25 GB mailboxes, the company had recognized through the analysis process of GroupWise that it is not in its best interest for users to store mail in individual mailboxes, but rather it should be migrated to Sharepoint in various forms better suited for accessing by a variety of users.

Many months of work have been summarized in this blog post, but there was more to come – once these changes were made we were ready to begin the actual migration from GroupWise to Office 365, the subject of our next blog post.

Advertisements
    • justinpirie
    • August 12th, 2011

    Great post!

    It’s so interesting to see how users use (and abuse?) email systems to hack them to their needs, without realising the impacts it can have on the oranisations and the costs it creates.

    Personally- I think even 25GB mailboxes are too big- I’d sooner see smaller mailboxes and then email archiving for long term retrieval- but I’m biased 😉

    I’m amazed that an organisation of this size, with SO MUCH mail hadn’t gone down that route in the past?

    Justin

  1. Users always find a way to be productive – they don’t believe they’re abusing the mail system, they’re just getting their work done.

    The consequences of their actions impact us as IT professionals and we have to find a way to make things work safely. If the whole mail system had failed, it still would have been our fault, not the users.

      • justinpirie
      • August 16th, 2011

      Absolutely spot on. We need to empower them with the tools to get their job done and be as much “out of the way” as possible.

      Let’s not trip ourselves up.

      The other side of the coin however is that mail systems have been typically been chronically underinvested in… so IT gets the blame even though they’ve had their hands tied…

    • Mika A
    • August 20th, 2011

    What’s also funny is the project as a whole. It started because “everyone wanted outlook”, yet they were using GroupWise for quite a lot more than plain email. Nevertheless, the decision was made to move to Outlook/Exchange and the project started, only after which it was revealed how extensively the Groupwise system was actually used. And due to this the whole “lets switch groupwise to exchange” -project changed to “dump GroupWise and buy/install Exchange, Sharepoint and Dynamics to get the functionality we now have”.

    So, in the end, instead of analyzing if there was something in the current system that could be done better or enhance it, they decided to change the whole system because email client was “not outlook”.

    Now I’m not saying the new system is not better than GroupWise alone was, I’m just questioning the reasoning and the decision process. GroupWise integrates to CRM (even to Dynamics) and
    Novell’s Vibe with Office integration is more than comparable to Sharepoint and brings quite a lot collaboration tools with it.

    It’s also a bit strange when you say “And if-and-when [servers] did crash, the users certainly didn’t see it as the fault of their giant mailboxes – it was just the IT department failing to do its job”.

    First, running 5000 GroupWise mailboxes on 60 servers sounds a bit strange, perhaps there are some explaining geographical reasons for that, but in general that’s about 10 times more than would be necessary.

    Second, how is it different to have a server crashing with GroupWise than it is with Sharepoint? Unless you use clustering or some other fault tolerant configuration a server crash is going to affect you. This may _seem_ different with/for the new system as it most likely is built with fault tolerance in mind.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: