(1 votes, average: 5.00 out of 5)
Loading...We recently took on the migration of our enterprise E-mail system from Exchange 2003 to 2010. In this article, I want to talk about the need to understand your client connection requirements before taking the plunge.
Like most companies, we are still on Outlook 2003 and Windows XP. Our remote users access the E-mail with RPC over HTTPS; Outlook over VPN, or other mobile devices using Microsoft Exchange ActiveSync, IMAP or Blackberry Enterprise Servers (BES). All of these need to be considered along with your corporate LAN users on Outlook. For the first time, we decided to use an integration partner to deploy Exchange 2010 rather than spend the time, money and resources to learn how to do it ourselves. While that is not uncommon, we learned a lot of hard lessons — the most important one being to ask lots and lots of questions. When doing something on your own, you spend time to read and learn, but when you hire this out, you expect that the company you hire has done this. Lesson here: Don’t assume!Everything Started with a BANG!
When we we reviewed the requirements with the outsource company, it seemed straight-forward to me. There was a fairly standard set of considerations:
- front-end/back-end Exchange 2003 servers
- Remote access via OWA, Microsoft ActiveSync, Blackberry Enterprise Server 5.0 and IMAP.
- All client systems have Outlook 2003.
- Resource calendars for meeting rooms
- Delegated authority to administrative assistants for executives
- lots of shared calendars so people know where others are
- public folders with varying levels of security
We met with the team and reviewed this along with our knowledge of the number of mailboxes, sizes of those mailboxes and our plans for growth. We built a very comprehensive model including access, storage, hardware, network, etc. Everything looked good to go and since they knew we were in production and wanted to migrate over time, we felt confident that they had the experience and would keep us moving forward.
Well, everything started off with a bang. We worked on the changes to Active Directory and began the installation of the software. We obtained the appropriate SAN certificates and things were moving faster and better than expected. Before one week was done, we had two virtual CAS servers, two MBOX servers in a DAG and 2010 OWA deployed for our web-based access users. We kept the blackberry users connected to 2003 Exchange, but everyone else was making connectivity through the new 2010 infrastructure while maintaining all mailboxes on Exchange 2003.
We were excited — and then the complaints came flooding in!
Before too long, the helpdesk was getting flooded with complaints and people were lining up at my desk. They could not access E-mail or at least it seemed intermittent. An analysis of the network traffic showed that the Microsoft Network Load Balancing (NLB) software was not maintaining sessions correctly and user’s were getting disconnected. After a long day and into the night we learn that there is a Kerberos authentication error when using NLB. How could that be? This team had recommended the use of NLB on our VMware virtual environment. Surely, they have done this before and would know of such an issue. It should have been a red flag that configuration of NLB on VMware 4.0 was research we had to do ourselves with the integration of networking on the Cisco 6500 switches. As it turns out, they had not done this before and just assumed (there’s that word again) that it would be OK. Now I know, anyone reading this article would say, “Steve, stop and think! You better regroup and see if you can move forward with this info”. And you would be right. But, I was thinking that I was already committed and the show must go on.
What we decided to do was force everything to NTLM. This would keep the clients connected. Uh Oh! What do you mean that the Outlook 2003 clients do not encrypt by default? Well, we had to set a group policy object (GPO) rule to force all Outlook 2003 clients to use NTLM. We pushed this through the domain and at least access issues were no longer an issue (so we thought).
Things went from bad to worse!
Next came the public folders. As I mentioned, we had a lot of them and they had a variety of security settings. We knew that 2010 does not install with support for public folders as Microsoft is trying to move that to SharePoint (we’ll talk about that in another article). But, there is still support for it in Exchange 2010. Our 3rd party team began the movement from 2003 to 2010 of this data and showed us the results of their work. We were pleased with the results and that all the security was intact. Then came the time for them to begin removing the replication of the public folders with 2003. They showed us the commands they would be executing and how it would leave us with all of the public folders in 2010, so I gave the green light to proceed. Within five minutes, it was clear there was a problem. All of the public folders were disappearing before our very eyes. Soon, they were history. I literally came unglued at this point. I was screaming at the outsource team and had executives from their company on the phone and reading them the riot act. I was furious over this huge error. The next day, I was a bit more calm and we discussed the recovery process. It took weeks to recover this data and we did it WITHOUT the outsource team. They proved to me that they were not to be trusted on this issue any more. Overall, that was a good decision as I would have spent a fortune on them to resolve a problem they caused.
Rather, I started them on the migration of users. I’ve done that before on test and on personal Exchange 2003 to 2010 environments. But those were simple compared to the enterprise world and again things began to unravel.
Problems With Calendars
The first users we migrated were IT staff and other low-level users. This worked fine and we could access all systems and services. Then came more feature-rich users and immediateley the phones started ringing. User’s complained that they could no longer see the shared calendars of certain users. Sure enough, it turns out that user’s on Exchange 2003 could not see the folders of users on Exchange 2010. Why not? Turns out that this is a know problem. Why weren’t we told? I found that our outsource company was trying to stay a step ahead of our complaints and looking up Google articles. Heck, I can do that and I am better at than they are. We were finding solutions ahead of them. Now I am getting really mad! We warn the users and apologize for the inconvenience (which is getting old now on this project).
So, we decide that administrative assistants better be completed before the executives they support. Turns out that was the right call since they are also delegates which would have opened a whole new can of worms.
Why Don’t My Messages Move?
The next issue came when an Outlook 2003 user tried to move a file from one folder to another one. The message “stayed” in the original folder. But, when they go to the destination folder it is there. When they go back to the source folder — it is now gone. Huh? In come the complaints and the line starts forming. When I ask the oursource company, they say, “Oh yeah. We know about that one”. You c an guess my blood pressure began getting dangerously high at this point. “When the heck were you going to tell me?”, I asked. So, we learned that Outlook 2003 is a polling-based environment that Exchange 2003 knew how to handle and updated clients automatically, but Exchange 2010 followed the polling wait-time. Then answer here is to create HKLM\CurrentControlSet\Services\MSExchangeRPC\Parameters\System\Maximum Polling Frequency” to something like 10000 on all CAS servers and ask all users to restart Outlook. Again, out goes the E-mail apology. It does not give immediate feedback they are used to, but we hope they can learn to deal with it until we release Outlook 2010.
Inconsistencies and Problems Continue
Users continue to have problems after migration. Seems they have lost their tasks or forms are missing or connection errors return. We soon learn that profiles are getting corrupted on their clients after the migration and we are having to go to desks and destroy Outlook 2003 profiles and recreate them for the users. This seems like a pure hack to me, but it gets our users back up and running after a short apology for the inconvenience.
User’s Still Lose Connectivity
We continue to fight over connectivity issues and I am convinced that using NLB on VMware with Exchange 2010 CAS servers is a really bad idea. We are going to try using Coyote Point 350 Load Balancers with virtual load balancing (VLB) technology. I’ll write about that when we get it going.
Blackberry and Cisco Unity
The last steps were to integrate Cisco Unity and the BES into a fully converted Exchange 2010 environment. Luckily, the documentation from Cisco and RIM is excellent. However, our migration of BES had far too many issues as we tried to change operating systems and other facets, but the technical support team at RIM had us rebuild the server and got us up and running without error within a few short hours.
Time To Say Goodbye to Exchange 2003 and our Outsource Company
Our contract with the outsource company was for them to do some of the first migrations and that we would take over after that. Then, they would return for the decommission of our Exchange 2003 environment. However, I was fed up with their lack of planning and knowledge. They might know Exchange 2010 really well, but they did not know how to effectively migrate and support a production migration. When they asked about the Exchange 2003 plans, I told them that we would do that and they could send me their final invoice. I paid them for their hours and we parted ways.
Lessons Learned
There were a few things we learned about this experience. First and foremost is to ask lots of questions to outsource companies about migration, technologies and get references. We trusted the recommendations we had without digging very deep. We checked on their knowledge of the new technology assuming they had a history of the old and migration issues.
If you are not going to go it alone, then do some research and see what problems or issues others have faced or their lessons learned. Ask you outsource team about these issues — but not directly on point. See if they have a playbook or their own set of items to review with you before you engage them. If they don’t have any checklists or concerns then that is your red flag to stop and rethink your approach.
Related articles
- Exchange 2003 to 2010 Migration (edugeek.net)
- Exchange 2003 to 2010 PreChecks (edugeek.net)
- Exchange 2010 with Outlook 2003 issues
