Azure Data Warehouse Now Available in Preview

Azure Data Warehouse is a new Microsoft service for staging large volumes of data for analysis purposes.  The Azure Data Warehouse service is now available in preview.

Azure Data Warehouse is a 100% PAAS based service to compliment the existing Azure SQL Database.  Azure DW is different than Azure SQL in a number of different ways:

  • Azure SQL is a traditional SQL database.  Azure Data Warehouse provides both relational and non-relational data processing models. 
  • Azure SQL is optimized for OLTP workloads where Azure Data Warehouse is optimized for ETL and analytics workloads.
  • Azure Data Warehouse is designed to be elastic to massive scale (e.g. Petabytes) where Azure SQL maxes out at 1 TB.
  • Azure Data Warehouse leverages Polybase which is Microsoft’s technology for integrating queries across SQL Server and Hadoop into a single model.

From a pricing perspective, there are some differences as well:

  • Azure SQL is sold based on performance units and so is Azure Data Warehouse.  However, these units are not the same.  It’s not clear which is faster or slower at the moment so it’s difficult to compare pricing from this perspective.
  • Azure SQL is priced on a monthly basis and cannot be stopped as a cloud service without deleting the database.  Azure Data Warehouse runs on demand and you can start and stop it.  Similar to an Azure VM, when stopped you only pay for storage costs and not processing costs.
  • With Azure SQL, you don’t pay for storage costs – they are bundled into the price.  With Azure Data Warehouse, you pay for storage costs for the data being housed in the warehouse.

Read More

Microsoft Partners with Akamai to Extend Reach of Azure CDN

The Azure Content Delivery Network is a service for caching and distributing content around the world.  For organizations with international traffic to their web site or applications, the Azure CDN pushes content such as video, images, pages or any other content to local nodes located closer to the end user to reduce latency and improve speed.

Here are the current locations of the Azure CDN nodes for distributing content around the world.

CDN POP Locations

Microsoft has just announced a partnership with Akamai to expand the Azure CDN network into Akamai’s CDN network.  Pricing has yet to be announced but Azure CDN will be moving to a two tier “Standard” and “Premium” model.

Akamai’s CDN platform already delivers 15-30% of the world’s web traffic through its massive network.  Akamai has more than 1000 node locations all over the world including South-America, Africa and the Middle East (which Microsoft currently has no presence).

Here is Akamai’s current network map – you can see that in particular in South America and

Read More

Microsoft Acquires ADX Studio to Enhance Dynamics CRM Offering

Microsoft has long suffered from a lack of a configurable portal layer for Dynamics CRM.  While the core product allowed you to design forms and custom entities for use by employees, developing self-service portals that were customer facing was not easily done without custom .NET code.

ADX Studio has been filling the gap by providing a third party solution that extends Dynamics CRM into self-service portals.  The solution is a .NET based solution that allows developers to build robust portals with authentication, data entry forms, surfacing of CRM data, etc. without the need for extensive development.  The solution reads the CRM schema dynamically so that if you change the underlying CRM model it is reflected automatically in the portal.

Microsoft has just acquired ADX Studio and will be integrating it into the core Dynamics product. Along with the recent acquisition of FieldOne, this means that gaps in the product that were previously being filled by third party applications will now be integrated as in house solutions.

Read More

Microsoft Azure Data Lake Service Expands Offering to Include Storage, Analytics and HD Insight

Microsoft previously announced a new Azure Data Lake service that would provide the repository for massively scalable big data storage.  The service has been refined and expanded to now include:

Azure Data Lake Store: what was previously announced as Azure Data Lake, provides a massively scalable repository for capturing data built on HDFS.

Azure Data Lake Analytics: a new service built on Apache YARN for performing distributed analytics.

Azure HD Insight: full managed Hadoop clusters, now available on Linux as well as Windows.

Azure Data Lake image

Read More

Visio Stencil for Microsoft Azure, Cloud and Enterprise

Microsoft has published a visio stencil you can download here that contains symbols for Microsoft Azure, Cloud and Enterprise.  

image

The stencil has some neat things in including a colour chart to guide you on Microsoft branding colours:

image

There is also an existing Office 365 Visio Stencil with symbols for all the various components that you can download here.


Read More

Massive Update to Power BI Desktop Just Released!

Microsoft has just updated Power BI Desktop with a bunch of new features and updates.  The full list of the 44 new features can be found here.  Here some of the more interesting features.

Drill Up and Down for Column and Scatter Charts

This is a key feature that was in Power View charts but missing from Power BI at launch.  You can now define column and scatter charts with drill down capabilities.

Calculated Tables

Calculated tables are virtual tables created from existing tables but filtered by a DAX expression.  For example, you could create a calculated table of just customers who are from New York.  Calculated tables work like regular tables, can be used in relationships and hierarchies and can be used to create dashboards.

Support for Querying Multiple Mailboxes

The Exchange Connector now supports queries for multiple mailboxes, each with their own credentials.   Using this new feature, user can combine data from multiple mailboxes into a single report.

Support for Inserting Shapes Into Reports

Users can now insert a shape such as a rectangle, oval, triangle or arrow into their reports.

Data Labels Formatting

You can now adjust the display units, font colors and data precision of data labels.

Support for Logarithmic Y Axis

In addition to the normal linear scale, you can now have a logarithmic scale for your Y Axis.

Read More

InfoPath 2013 Now Available as Stand Alone Download

With the release of Office 2016, there is no InfoPath version available since InfoPath 2013 is the last available release.  Microsoft has now released a standalone version of InfoPath 2013 for those users who have Office 2016 but still want to use InfoPath without need to install a full Office 2013.

You can find the download here.

Read More

New Invite App Makes Finding Meeting Time Easier

As part of Microsoft’s “Garage” series of apps, there is a new mobile app for organizing meetings.  One of the key challenges for organizing any meeting is finding a time where all the participants are free.  The new Invite app allows you to organize a meeting by sending out a number of possible dates and times.  Your attendees can then vote on which times they are available and you can then pick the best time based on their feedback.

Invite 1Invite 2

The app is currently available for IPhone and coming soon for Android and Windows Phone.

Read More

Integrating SharePoint with Microsoft’s new Computer Vision API

Microsoft has released a  new API called the Computer Vision API.  It provides OCR conversion from images to text.  Similar to the recent post on integration Microsoft’s new Face API, I wanted to test whether you could integrate SharePoint with this new API to act as an auto-tagging mechanism for images stored in document libraries.

Microsoft has also provided a C# and Android SDK for the Computer Vision API that you can download here.

I wanted to develop a demo that leveraged the document management power of SharePoint and integrate it with the OCR capabilities of this new API.  The demo code I’m going to describe is posted to github here.

The Scenario

Imagine you have a document library filled with images that could have text on them.  If you try to search for these images, you won’t easily find them because they have no metadata related to the content within the image.  Could we use an OCR service to scan the image and pull out the text so we could update the metadata with the found text?

Step #1: Establishing a Domain Model

I wanted a class to represent each photo.  This allows us to populate a list of photos from SharePoint, hand the list off to another class responsible for tagging the photos and then hand it back to the SharePoint service to update SharePoint back again.  Here is the class definition.

/// <summary>
   /// Value object representing a Photo. 
   /// </summary>
   public class Photo
   {
       public byte[] Image { get; set; }
       public string ID { get; set; }
       public List<string> TextInPhoto { get; set; }
       public string LanguageDetectedInPhoto { get; set; }

     public int NumberOfMatchedFaces { get; set;  }

     public int NumberOfUnmatchedFaces { get; set; }

     public List<PhotoPerson> PeopleInPhoto { get; set; }

     public Photo()
       {
           PeopleInPhoto = new List<PhotoPerson>();
           TextInPhoto = new List<string>();
       }

}

Step #2: Pulling Images from SharePoint

Using the SharePoint client APIs (CSOM) with C#, I wrote a method to pull images out of a target document library.

public List<Photo> getPhotosToTag()
          {
              List<Photo> photos = new List<Photo>();


            using (ClientContext context = Login(SharePointURL))
              {
                  try
                  {
                      var list = context.Web.GetList(PhotosToTagURL);
                      var query = CamlQuery.CreateAllItemsQuery();


                    var result = list.GetItems(query);
                      ListItemCollection items = list.GetItems(query);
                      context.Load(items, includes => includes.Include(
                          i => i[PhotoFileColumn],
                          i => i[PhotoIdColumn]));


                    //now you get the data
                      context.ExecuteQuery();


                     //here you have list items, but not their content (files). To download file
                      //you’ll have to do something like this:


                    foreach (ListItem item in items)
                      {
                          Photo photo = new Photo();
                          //get the URL of the file you want:
                          var fileRef = item[PhotoFileColumn];


                        //get the file contents:
                          FileInformation fileInfo = Microsoft.SharePoint.Client.File.OpenBinaryDirect(context, fileRef.ToString());


                        using (var memory = new MemoryStream())
                          {
                              byte[] buffer = new byte[1024 * 64];
                              int nread = 0;
                              while ((nread = fileInfo.Stream.Read(buffer, 0, buffer.Length)) > 0)
                              {
                                  memory.Write(buffer, 0, nread);
                              }
                              memory.Seek(0, SeekOrigin.Begin);


                            photo.ID = item.Id.ToString();
                              photo.Image = memory.ToArray();
                              photos.Add(photo);
                          }


                    }
                  }
                  catch (Exception e)
                  {
                      throw;
                  }
              }


   return photos;
}

The important information we need to track is the ID of the photo so that we can update it back into SharePoint once we’re finished processing.

Step #3: Sending Photos to the Computer Vision API

Once we have our list of photos, we can now send these to the Computer Vision API for text identification.

public async Task identifyTextInPhoto(List<Photo> Photos)
          {
              try
              {
                  foreach (Photo photo in Photos)
                  {
                      VisionServiceClient client = new VisionServiceClient(SubscriptionKey);
                      Stream stream = new MemoryStream(photo.Image);
                      OcrResults result = await client.RecognizeTextAsync(stream, Language, DetectOrientation);
                      photo.LanguageDetectedInPhoto = result.Language;
                      foreach (Region region in result.Regions)
                      {
                          for (int i=0; i< region.Lines.Length; i++)
                          {
                              Line line = region.Lines[i];
                              string lineText = “”;
                              for (int j= 0; j < line.Words.Length; j++)
                              {
                                  lineText += line.Words[j].Text;
                                  if (j < line.Words.Length -1)
                                  {
                                      lineText += ” “;
                                  }
                              }
                              photo.TextInPhoto.Add(lineText);
                          }
                      }


                }
              }
              catch (Exception e)
              {
                  throw;
              }
          }

This method call uses the supplied Computer Vision API SDK to send each image to the OCR service and pulls in the found text.

Step #4: Sending the Results Back to SharePoint

When the Computer Vision API analyzes your image, it returns back text as a series of regions, lines and words.  The API provides not only the text but the bounded rectangle where it was found.

public async Task identifyTextInPhoto(List<Photo> Photos)
       {
           try
           {
               foreach (Photo photo in Photos)
               {
                   VisionServiceClient client = new VisionServiceClient(SubscriptionKey);
                   Stream stream = new MemoryStream(photo.Image);
                   OcrResults result = await client.RecognizeTextAsync(stream, Language, DetectOrientation);
                   photo.LanguageDetectedInPhoto = result.Language;
                   foreach (Region region in result.Regions)
                   {
                       for (int i=0; i< region.Lines.Length; i++)
                       {
                           Line line = region.Lines[i];
                           string lineText = “”;
                           for (int j= 0; j < line.Words.Length; j++)
                           {
                               lineText += line.Words[j].Text;
                               if (j < line.Words.Length -1)
                               {
                                   lineText += ” “;
                               }
                           }
                           photo.TextInPhoto.Add(lineText);
                       }
                   }


             }
           }
           catch (Exception e)
           {
               throw;
           }
       }
   }

For example, this image returns one region with three lines of text composed of words in each line.

Quebec_License_Plate

In this simple example, we treat each line found as a new line and separate each word with a space (if punctuation is found, the API adds it to the word automatically.

Once we have our matched text, we can update our original SharePoint list item with the found text.

public void updateTaggedPhotosWithText(List<Photo> Photos)
          {
              using (ClientContext context = Login(SharePointURL))
              {
                  try
                  {
                      foreach (Photo photo in Photos)
                      {
                          SP.List list = context.Web.GetList(PhotosToTagURL);
                          ListItem item = list.GetItemById(photo.ID);
                         
                          string textInPhoto = “”;


                        string[] lines = photo.TextInPhoto.ToArray();


                        for (int i = 0; i < lines.Length; i++)
                          {
                              textInPhoto += lines[i];
                              if (i < lines.Length – 1)
                                  textInPhoto += “\n”;
                          }
                          item[PhotoTextColumn] = textInPhoto;
                          item.Update();
                          context.ExecuteQuery();
                      }
                  }
                  catch (Exception e)
                  {
                      throw;
                  }
              }


        }

Conclusion

Some of the image processing in the current Computer Vision API worked really well while other images failed to find the right text.   Keep in mind these APIs are still in beta and OCR is a notoriously difficult process to perfect especially with general image analysis.

For example, this license plate (found randomly on Google images) worked:

Quebec_License_Plate

but this one did not find the plate in the middle.

on2010jan

The Computer Vision API seems to work reasonably well with type faces, but doesn’t work at all with hand written text.  These images for example did not return any text.

img_9036

handwriting

This image returned Midnight but not Show.

poster1-o

Read More

Integrating SharePoint and Microsoft’s New Face Identification APIs

Microsoft has a new set of APIs in beta for recognizing faces.  You can sign up to leverage these APIs through the Azure Marketplace. 

image

This provides your account with a subscription key you can use to send data to the API.  The API is REST based but there is also a .NET SDK that wraps the REST layer within .NET objects.   

I wanted to develop a demo that leveraged the document management power of SharePoint and integrate it with the face identification capabilities of this new API.  The demo code I’m going to describe is posted to github here.

The Scenario

Many organizations have pictures of people in SharePoint, from company photos, pictures of customers, etc.  These pictures are typically uploaded into image libraries and then manually tagged.  The tags allow for searching – without a tag, SharePoint cannot filter or refine the search to find pictures of a specific person. 

Could we write a piece of software that would grab the pictures from a document library, send the images to the Microsoft Face API, identify the faces found in the pictures and then update the tags automatically?  With a little bit of coding, I built such a demo solution.

Step #1: Establishing a Domain Model

Using a Domain Driven Design approach, I wanted a set of classes that could act as the domain model.  For this domain, we need two basic objects: 1) a person and 2) a photo.  The person and photo have a many to many relationship, e.g. a person can have a list of photos and a photo can have multiple faces in it and link to multiple people.

/// <summary>
  /// Value object representing a Photo. 
  /// </summary>
  public class Photo
  {
      public byte[] Image { get; set; }
      public string ID { get; set; }
     
      public List<string> TextInPhoto { get; set; }
      public string LanguageDetectedInPhoto { get; set; }


     public int NumberOfMatchedFaces { get; set;  }


     public int NumberOfUnmatchedFaces { get; set; }


     public List<PhotoPerson> PeopleInPhoto { get; set; }


     public Photo()
      {
          PeopleInPhoto = new List<PhotoPerson>();
          TextInPhoto = new List<string>();
      }


}

public class PhotoPerson
{
     public string Name { get; set; }
     public int ID { get; set; }
     public List<Photo> Photos { get; set; }


    public PhotoPerson(int ID)
     {
         this.ID = ID;
         Photos = new List<Photo>();
     }


    public PhotoPerson()
     {
         Photos = new List<Photo>();
     }


    public PhotoPerson(int ID, string Name)
     {
         this.ID = ID;
         this.Name = Name;
         Photos = new List<Photo>();
     }

}

The benefit of this approach is we have a neutral, non-SharePoint specific domain model that we can pass data from the SharePoint service responsible for pulling and pushing data in and out of SharePoint and the FaceTagger service which is responsible for pulling and pushing data into the Microsoft Face API.  Neither class is now dependent on the other and this makes unit testing very simple and easy.

Step #2: Creating a Training Set

The first step in creating a matching algorithm is to create a training set.  The training set is our master list of photos that we know are good.  In this case, I used a picture of myself and my two children as a training set, stored in a SharePoint Image library.

image

You can have multiple images for the same person if you wish and this will make the training set more effective at matching by combining and averaging characteristics found in multiple photos.

In order to pull the data from SharePoint, I used the client side SharePoint APIs (CSOM) to fetch the images from the library.  

public Dictionary<string, PhotoPerson> getTrainingPhotos()
         {
             Dictionary<string, PhotoPerson> trainingPhotos = new Dictionary<string, PhotoPerson>();


            using (ClientContext context = Login(SharePointURL))
             {
                 try
                 {
                     var list = context.Web.GetList(TrainingListURL);
                     var query = CamlQuery.CreateAllItemsQuery();
                    
                     var result = list.GetItems(query);
                     ListItemCollection items = list.GetItems(query);
                     context.Load(items, includes => includes.Include(
                         i => i[TrainingPersonIdColumn],
                         i => i[TrainingFileColumn],
                         i => i[TrainingIdColumn]));
                        
                        


                    //now you get the data
                     context.ExecuteQuery();


                    //here you have list items, but not their content (files). To download file
                     //you’ll have to do something like this:


                    foreach (ListItem item in items)
                     {
                         PhotoPerson person = null;
                         if (item[TrainingPersonIdColumn] != null)
                         {
                             string fullName = (string)item[TrainingPersonIdColumn];
                             // look for existing person
                             if (trainingPhotos.ContainsKey(fullName))
                             {
                                 person = trainingPhotos[fullName];
                             }
                             else
                             {
                                 person = new PhotoPerson();
                                 person.Name = fullName;
                                 person.ID = item.Id;
                             }


                            //get the URL of the file you want:
                             var fileRef = item[TrainingFileColumn];


                            //get the file contents:
                             FileInformation fileInfo = Microsoft.SharePoint.Client.File.OpenBinaryDirect(context, fileRef.ToString());
                            
                             using (var memory = new MemoryStream())
                             {
                                 byte[] buffer = new byte[1024 * 64];
                                 int nread = 0;
                                 while ((nread = fileInfo.Stream.Read(buffer, 0, buffer.Length)) > 0)
                                 {
                                     memory.Write(buffer, 0, nread);
                                 }
                                 memory.Seek(0, SeekOrigin.Begin);
                                 Photo photo = new Photo();
                                 photo.ID = item.Id.ToString();
                                 photo.Image = memory.ToArray();
                                 person.Photos.Add(photo);
                             }


                            trainingPhotos.Add(fullName, person);


                        }
                        
                     }



                }
                 catch (Exception e)
                 {
                     throw;
                 }
             }
             return trainingPhotos;


        }

The important part is we need to have photos organized by person – we can do this by using a field such as full name as the identifier to link photos together.  The result of this is a Dictionary collection of people and their related photos. 

Once we have our training photos loaded, we can push them to Microsoft’s Face API.  This done through the Face Client SDK.

public async Task addPhotosToTrainingGroup(Dictionary<string, PhotoPerson> Photos, string PersonGroupID)
{
     IFaceServiceClient faceClient = new FaceServiceClient(SubscriptionKey);


    // Get the group and add photos to the group.
     // The input dictionary is organized by person ID.  The output dictionary is organized by the GUID returned by the added photo from the API.
     try
     {
         await faceClient.GetPersonGroupAsync(PersonGroupID);


        // training photos can support multiple pictures per person (more pictures will make the training more effective). 
         // each photo is added as a Face object within the Face API and attached to a person.


        foreach (PhotoPerson person in Photos.Values)
         {
             Person p = new Person();
             p.Name = person.Name;
             p.PersonId = Guid.NewGuid();


            List<Guid> faceIDs = new List<Guid>();


           
             foreach (Photo photo in person.Photos)
             {
                 Stream stream = new MemoryStream(photo.Image);
                 Face[] face = await faceClient.DetectAsync(stream);


                // check for multiple faces – should only have one for a training set.
                 if (face.Length != 1)
                     throw new FaceDetectionException(“Expected to detect 1 face but found ” + face.Length + ” faces for person ” + p.Name);
                 else
                     faceIDs.Add(face[0].FaceId);
             }


            Guid[] faceIDarray = faceIDs.ToArray();


            // create the person in the training group with the image array of faces.
             CreatePersonResult result = await faceClient.CreatePersonAsync(PersonGroupID, faceIDarray, p.Name, null);
             p.PersonId = result.PersonId;
             TrainingPhotos.Add(p.PersonId, person);


        }


        await faceClient.TrainPersonGroupAsync(PersonGroupID);
         // Wait until train completed
         while (true)
         {
             await Task.Delay(1000);
             var status = await faceClient.GetPersonGroupTrainingStatusAsync(PersonGroupID);
             if (status.Status != “running”)
             {
                 break;
             }
         }
     }
     catch (ClientException ex)
     {
         throw;
     }


}

This method creates a set of Person and Face objects, links them together and pushes them up to the Face API.  Once the whole set is uploaded, we tell the Face Client to run the training algorithm which analyzes these photos to create a master reference set of faces for identification.

Step #3: Identifying and Matching Faces

Now that we have a training set, we can identify faces and match them against our master training set.  I uploaded some unidentified pictures to test out the API and see if the Face API could recognize the people in the pictures.

image

In this SharePoint list, I created the following columns to store the output of the matching algorithm:

  • Number of Matched Faces
  • Number of Unmatched Faces
  • Matched People

In a similar way to Step #2, we need a method for fetching the untagged images from SharePoint.

public List<Photo> getPhotosToTag()
         {
             List<Photo> photos = new List<Photo>();


            using (ClientContext context = Login(SharePointURL))
             {
                 try
                 {
                     var list = context.Web.GetList(PhotosToTagURL);
                     var query = CamlQuery.CreateAllItemsQuery();


                    var result = list.GetItems(query);
                     ListItemCollection items = list.GetItems(query);
                     context.Load(items, includes => includes.Include(
                         i => i[PhotoFileColumn],
                         i => i[PhotoIdColumn]));


                    //now you get the data
                     context.ExecuteQuery();


                     //here you have list items, but not their content (files). To download file
                     //you’ll have to do something like this:


                    foreach (ListItem item in items)
                     {
                         Photo photo = new Photo();
                        
                         //get the URL of the file you want:
                         var fileRef = item[PhotoFileColumn];


                        //get the file contents:
                         FileInformation fileInfo = Microsoft.SharePoint.Client.File.OpenBinaryDirect(context, fileRef.ToString());


                        using (var memory = new MemoryStream())
                         {
                             byte[] buffer = new byte[1024 * 64];
                             int nread = 0;
                             while ((nread = fileInfo.Stream.Read(buffer, 0, buffer.Length)) > 0)
                             {
                                 memory.Write(buffer, 0, nread);
                             }
                             memory.Seek(0, SeekOrigin.Begin);


                            photo.ID = item.Id.ToString();
                             photo.Image = memory.ToArray();
                             photos.Add(photo);
                         }


                    }
                    
                 }
                 catch (Exception e)
                 {
                     throw;
                 }
             }

     return photos;
}

The important information we need to track is the ID of the photo so that we can update it back into SharePoint once we’re finished processing.

Once we have our list of photos, we can now send these to the Face API for identification.

public async Task identifyPhotosInGroup(string PersonGroupID, List<Photo> Photos)
         {
             IFaceServiceClient faceClient = new FaceServiceClient(SubscriptionKey);
                        
             try
             {
                 foreach (Photo photo in Photos)
                 {
                     photo.NumberOfMatchedFaces = 0;
                     photo.NumberOfUnmatchedFaces = 0;
                     photo.PeopleInPhoto.Clear();


                    // convert image bytes into a stream
                     Stream stream = new MemoryStream(photo.Image);


                    // identify faces in the image (an image could have multiple faces in it)
                     var faces = await faceClient.DetectAsync(stream);


                    if (faces.Length > 0)
                     {
                         // match each face to the training group photos. 
                         var identifyResult = await faceClient.IdentifyAsync(PersonGroupID, faces.Select(ff => ff.FaceId).ToArray());
                         for (int idx = 0; idx < faces.Length; idx++)
                         {
                             var res = identifyResult[idx];
                             if (res.Candidates.Length > 0)
                             {
                                 // found a match so add the original ID of the training person to the photo
                                 if (TrainingPhotos.Keys.Contains(res.Candidates[0].PersonId))
                                 {
                                     photo.PeopleInPhoto.Add(TrainingPhotos[res.Candidates[0].PersonId]);
                                     photo.NumberOfMatchedFaces += 1;
                                 }
                                 // didn’t find a match so count as an unmatched face.
                                 else
                                     photo.NumberOfUnmatchedFaces += 1;
                             }
                             // didn’t find a match so count as an unmatched face.
                             else
                                 photo.NumberOfUnmatchedFaces += 1;


                        }
                     }


                }


             }
             catch (ClientException ex)
             {
                 throw;
             }



        }

The Face API takes each photo, identifies the faces in the photo and then identifies candidates from the training set that could be a match.  Once a match is found, we link the photo to the original person from the training set.

Step #4: Tagging Photos in SharePoint

Now that we have our matches identified, we can update our photos in SharePoint with the new information.

public void updateTaggedPhotosWithMatchedPeople(List<Photo> Photos)
         {
             using (ClientContext context = Login(SharePointURL))
             {
                 try
                 {
                     foreach (Photo photo in Photos)
                     {
                         SP.List list = context.Web.GetList(PhotosToTagURL);
                         ListItem item = list.GetItemById(photo.ID);
                         item[PhotoNumberOfFacesColumn] = photo.NumberOfMatchedFaces;
                         item[PhotoNumberOfUnMachedFacesColumn] = photo.NumberOfUnmatchedFaces;

                        FieldLookupValue[] matchedPeople = new FieldLookupValue[photo.PeopleInPhoto.Count];
                         for (int i=0; i< photo.PeopleInPhoto.Count; i++)
                         {
                             FieldLookupValue value = new FieldLookupValue();
                             value.LookupId = photo.PeopleInPhoto[i].ID;
                             matchedPeople[i] = value;
                         }
                         item[PhotoMatchedPeopleColumn] = matchedPeople;
                         item.Update();
                         context.ExecuteQuery();
                     }
                 }
                 catch (Exception e)
                 {
                     throw;
                 }
             }

        }

The trickiest part is updating the lookup column with the person matched.  Remember that for each photo we could have multiple people so we need to support multiple values in the column.  The SharePoint CSOM API provides the ability to perform such an update using a special object called a FieldLookupValue.  In addition, we update the list with the number of matched faces and the number of unmatched faces.

Here is the result of the whole process!

image

Conclusion

The Face API while in beta is still pretty good at matching faces.  All of these pictures are from the same family members and Microsoft Face API still was able to distinguish between me, my daughter Katie and my son Geoffrey.  I also tried uploading a picture of Katie when she was about 10 years younger and the Face API was able to still identify her successfully.  The third picture has all three of us huddle together and the Face API was able to identify all three of us distinctly.

However, there were a few photos that the Face API struggled with in identifying faces.

11334232_10155742753290501_1341653627466259388_o

This one was identified as not a match even though it is a match.

This picture the Face API failed to identify any face at all.

11822353_10155915186250501_8444461154400401535_n

So clearly there is some work to do in the research to achieve near perfect results.  However, for corporate contexts where pictures tend to be more formal the algorithm seems to work really well.

Please note also that the current Face API only supports 20 transactions per minute and 5000 transactions per month, so you won’t be able to run this program across hundreds of images – if you try you’ll get an exception.  The API is still in the research stage so it isn’t available as a production service yet.

Read More