Author Archives: John Samuel

Pitfalls of migrating to SharePoint on Microsoft Office 365

We recently had to migrate an in house file system to SharePoint in the cloud. The client had over 100GB of data and close to 200,000 files and thousands of folders. We had done a lot of work with SharePoint in a locally hosted environment and the following is our experience with migrating a physical environment to SharePoint online.

It helped that they were an existing Office 365 client for email. We did all the basic legwork and add SharePoint to their existing account.

We initially started with using the client server to try to copy this massive amount of data. The following should be carefully evaluated;

  1. Is the local internet connection fast enough?
  2. Does the client have the latest version of IE as required for SharePoint?
  3. Does the client have more than 5,000 files per folder? If they do look at SharePoint requirements very carefully. Refer to Software boundaries and limits for SharePoint 2013 –technet.microsoft.com/en-us/library/cc262787.aspx
  4. Does the client have files and folders with illegal characters? And don’t think illegal characters are what you think they are. We’ll get into details on this later. http://support.microsoft.com/kb/905231
  5. What is your migration plan?
  6. How will you know when you are done?

Is your Internet Speed fast enough?

If the internet speed is slow or goes down intermittently you will have issues trying to get the data copied fast enough. If the copy is interrupted and you don’t have a way to continue where you left off then you will have a hell of time trying to synchronize the data.

How should I copy data?

The best way to start is to plan, plan and plan. Don’t just jump in and think it will just work. Plan out what you will do while the data is copying and users are modifying original documents.

Thin k about copying the data to a local PC with up to date Windows and Internet Explorer installed. SharePoint and Internet Explorer will allow you to open a SharePoint library in a windows explorer screen to make life easy for diagnostics and progress monitoring.

If you think you can use a good utility such as terracopy, robocopy, kdiff, synctoy and anything else you may normally use then you will be pleasantly surprised.

The files copied to SharePoint do not retain the same file size and they present as different sizes once they are copied which makes comparing files very difficult. You can set your compare to ignore file size but then you risk skipping over files that were improperly or partially copied.

A lot of these utilities will also not work due the number of disallowed conditions. See a list later in this article.

When you are ready to give up try MetaVis?

Needless to say we ended up giving up on any standard utility that would have worked in any other normal migration. After much research we ended up going for a product from MetaVis Technologies (metavis.com). The product was easy to get started with. The price was very economical at  around $595. This version is hard to find on their website but is called Information Manager – Power User Edition (http://www.metavistech.com/product/information-manager-power-user-edition) which will allow it to be used on a single PC. The product will allow you to drag and drop files from your local system to SharePoint. It will also handle the required renaming of folders and files on the fly. We did experience a lot of problems with files failing but tech support was great and provided a lot of help. During our ordeal they provided a couple of updates to the products. At the end it would have been nearly impossible to get the project completed without the product.

How do you know when you are done?

You really don’t MetaVis did not have a great way to compare folders and the only way to make sure we didn’t miss anything in the massive file list of nearly 200,000 files was to drag the files to SharePoint many times to deal with interrupted copies due to internet glitches, power causing the PC to reboot and many more. Unfortunately MetaVis also did not have a progress bar to indicate where it was in the copy process. This also hindered our ability to monitor what was going on. Microsoft and the Office 365 portal was useless in figuring out where we were in the copy process.

The best tool proved to be the good old way of mapping a drive and doing right-click, properties to see size on disk and watch it get closer to our target size. As mentioned earlier since the file size in your local file system and SharePoint differ and due to many illegal file types, it’s nearly impossible to match local vs. target folders.

Bottom line is you may never know you are done. You may end up setting a target to be close on more recent or important folders but some missing files may be ok. In case you are planning to decommission the local file system, make sure you copy the data to an external hard drive and provide it to an entrusted individual in the company and if files are missing down the road they can be individually copies.

Here is a list of what is not allowed per Microsoft KB905231.

Information about the characters that you cannot use in site names, folder names, and file names in SharePoint, KB905231. There are many others besides these, such as exe, dll, and many others. See http://office.microsoft.com/en-us/windows-sharepoint-services-help/types-of-files-that-cannot-be-added-to-a-list-or-library-HA010100147.aspx

Site names, subsite names, or site group names

  • You cannot use the following characters anywhere in a site name, in a subsite name, or in a site or Active Directory group name:
    • Tilde (~)
    • Number sign (#)
    • Percent (%)
    • Ampersand (&)
    • Asterisk (*)
    • Braces ({ })
    • Backslash (\)
    • Colon (:)
    • Angle brackets (< >)
    • Question mark (?)
    • Slash (/)
    • Plus sign (+)
    • Pipe (|)
    • Quotation mark (“)
  • You cannot start a site name, a subsite name, or a site group name with an underscore (_) character or with the period (.) character.
  • When you create a site name, a subsite name, or a site group name, you cannot use strings that were already used to name managed paths.
  • You cannot use the period character consecutively in the middle of a site name, a subsite name, or a site group name.
  • You cannot use the period character at the end of a site name, a subsite name, or a site group name.

Folder names

  • You cannot use the following characters anywhere in a folder name or a server name:
    • Tilde
    • Number sign
    • Percent
    • Ampersand
    • Asterisk
    • Braces
    • Backslash
    • Colon
    • Angle brackets
    • Question mark
    • Slash
    • Pipe
    • Quotation mark
  • You cannot use the period character consecutively in the middle of a folder name.
  • You cannot use the period character at the end of a folder name.
  • You cannot start a folder name with a period character.

File names

  • You cannot use the following characters anywhere in a file name:
    • Tilde
    • Number sign
    • Percent
    • Ampersand
    • Asterisk
    • Braces
    • Backslash
    • Colon
    • Angle brackets
    • Question mark
    • Slash
    • Pipe
    • Quotation mark
  • You cannot use the period character consecutively in the middle of a file name.
  • You cannot use the period character at the end of a file name.
  • You cannot start a file name by using the period character.
  • In addition, file names and folder names may not end with any of the following strings:
    • .files
    • _files
    • -Dateien
    • _fichiers
    • _bestanden
    • _file
    • _archivos
    • -filer
    • _tiedostot
    • _pliki
    • _soubory
    • _elemei
    • _ficheiros
    • _arquivos
    • _dosyalar
    • _datoteke
    • _fitxers
    • _failid
    • _fails
    • _bylos
    • _fajlovi
    • _fitxategiak

What is the difference between VHD or VHDX files

Microsoft HyperV allows you to create 2 primary hard disk storage types, fixed or dynamic.

Dynamic size drives are allocated with the minimum space needed and space is expanded as you write additional drive to the volume. There is performance loss on disk writes that require storage to be expanded. For example if you create a dynamic volume with 10 GB and as you write data and use up the entire 10GB allocation then the volume needs to be expanded first then your data will be written to the volume. This expansion adds overhead and can slow down data writes by 25% or more.

Fixed drive sizes presets the size for your storage. Since the file is preallocated on the HyperV host there is no loss in performance due to the need for expansion of space.

The big difference between VHD and VHDX is the way this expansion is handled. In tests performed by others they have seen negligible difference in overhead with Dynamic VHDX volumes when they expand to accommodate additional data.

You can easily convert your VHD volumes to VHDX. As always, make a backup and make sure you have enough time allocated for the conversion. The larger the volume the longer the conversion will take.

Is the PC almost dead?

Microsoft and Intel are doing their best to save the PC industry. The Computex trade show in Taiwan will feature new form factors utilizing the latest releases from Intel and a the version of Microsoft’s Windows, Windows 8.1. Wintel lives on. We have seen the onslaught of Windows 8 devices that has opened the door for evolution of the existing laptop and tablet devices. There are now many new hybrid laptops, touchscreen laptops and convertible tablet devices. The netbook was short lived and now tablets are positioned to pass PC sales a few short years.

Intel’s Haswell platform is planned to be available in June or July. The next generation CPUs are aimed primarily at laptop and hybrid tablet manufacturers, offering increased battery life and performance. Desktop users can also expect a considerable boost in power. PC makers hope for a boost once Windows 8.1 aka Windows Blue is paired with the Haswell based technology. Intel claims the new chips will offer “double the graphics performance” for laptops, and triple the performance for desktop PCs. Reportedly, the Haswell chips will reduce power consumption to 7 watts; a significant leap from the 10 watt Ivy Bridge Core processors.

According to IDC, “the U.S. market came in slightly weaker than an already negative forecast, contracting 12.4% compared to a forecast of –9.5%. This reflected weaker consumer demand, including a weak back-to-school season, and an industry-wide inventory clean up.“

A Windows 8.1 refresh could be holding the supply chain back while people wait for their START button to be put back. Windows 8.1 or Win Blue is a very aggressively roll out for Microsoft. The new Win Blue is expected to resolve some of the drawbacks with Win8, bringing back the “Start” button and provide better adoption by users used to previous versions of Windows.

The demise of the PC based technology is not as close as everyone would like to theorize. At least not until tablets and hybrids evolve enough for them to handle the needs of the business users. We still love our big screens, keyboard and mice.