SPC125 – Hybrid and Search in the Cloud – Brad Stevenson

Brad Stevenson talks about creating a search experience which spans an On-Premises and Office365 SharePoint 2013 environment.

Search in the cloud

So what is the story about the cloud search capability?

Comparing SP2010 online to SP2013 online

• Crawl freshness is important for any search system as its important that a user trusts the results they are given. Part of this trust is that they are seeing the latest content within a timely window.
• Query latency is already really quite snappy in SP2010 online. SP2013 moves to client side rendering approaches to improve this snappiness perception even further. This approach allows the server to share some of the rendering load with the client device improving performance for the end user.
• Scale in SP2013 originates from the FAST technologies so brings those benefits to bear, making it a powerful and scalable platform solution.
• Manageability within SP2013 allows more control over the schema, examples are the control over managed properties and result sources. A lot of the features which were part of the service application have now been brought down to the site collection and tenant administration levels.
• User Experience is dramatically different with new capabilities such as hover panels, visual refinements etc. This helps a user to establish the relevance of a result without leaving the results page or downloading the documents.
• Extensibility is improved without writing code with such elements such as the rendering templates replacing the complex XSL.

Search extensibility in the cloud

No code:

• Managed properties
• Result sources
• Query rules
• Result types
• Display templates
• Dictionaries (via the Term Store)

Code:

• CSOM
• REST

Packaging:

• Import/Export search settings
• SharePoint apps

You manage your ‘global search’ via the tenant admin interface. The only major piece of the service application settings you have no control over is the crawl scheduling. In a multi-tenant environment this really makes sense.

Journey to the cloud

Definition of the cloud?

• Public cloud – Office 365, allows you to focus on just the software services.
• Private cloud – Windows Azure, allows you to offload the OS and hardware to the cloud provider.
• Traditional – All managed by the internal organisation

Moving to the cloud

What to move? (not just everything) and should it be everything including customisations and settings.

When to move it? How do you plan the move? Is it an all or nothing or staged co-existence.

How to move it? What tools are available?

The migration lifecycle

Early – 90% on-prem 10% cloud

Mid – 50% on-prem 50% cloud

Late – 10% on-prem 90% cloud

How hybrid search can help

User want to easily find content, they just want to find things they’re looking for and not have to think about understanding the systems structure. It is about getting their job done efficiently.

Users don’t care about migration. So don’t force users to track what’s being moved and when.

Realise that most users will never move EVERYTHING to the cloud.

Hybrid Search User Experience

Demo environment details:

• On-Premises SP2013 crawling a mixture of SharePoint data and file shares
• Office365 indexing all of its content
• Firewall between

So the idea is that from within either environment the user can get results from either. They use query rules to ‘cross-pollinate’ results from the other environment as a block of results. Personally I’m not sure this is a great user experience. It gives a false impression to a user of which results are most important. So I remain to be convinced about using result blocks.

A neat thing to know is that the refinement panel operates over ALL returned results rather than just the local SharePoint items. Also the hover panels are dependant on the sources WOPI configuration.

Configuring Hybrid Search

Configuration steps:

• Choose one-way or bi-directional
• Deploy and configure pre-reqs
• Configure search data settings
• Configure search UI settings

If you are early on in your migration lifecycle a one-way where on-premises indexes Office365 might suit your needs. Or late on a one-way works for Office 365 to use on-premises. Mid-life is definitely bi-directional where the experience should be the same.

Environment Configuration

One-way or Bi-directional

Where will users go to search?

• Just on-premises
• Just Office365
• Both

Hybrid pre-requisites

Non-SharePoint:

• Reverse proxy and certificate authentication
• Identity provider (ADFS or Shibboleth for Office365)
• MSOL Tools
• SSO with Office365
• DirSync

SharePoint:

• New SharePoint STS Token signing certificate
• Configure a trust relationship between SharePoint on-premises and ACS
• Configure secure store
• Configure UPA

Configure Data Settings

Result source (equivalent to a federated location and scope in SP2010) pointing at:

• URL of remote location
• Secure Store (for client certificate)

Configure UI Settings

• Query rule to show remote results in ‘Everything’ tab
• Search vertical which ‘only’ displays results from remote location (Office365 or on-premises)

Search Centre On-Premises: Data Flow

Scenario One:

User logs into on-premises and issues a search query. It actually issues two queries. First is to the local on-premises index. The second is issued to Office365. The second query is issued through the CSOM endpoint within Office365. Identity mappings take place where the on-premises identity is mapped to the Office365 identity. Office365 then performs the query and issues the results response.

Search Centre in Office365: Data Flow

Scenario Two:

Basic flow is a reverse of on-premises except there is the introduction of the revers proxies at the perimiter to route the request back to the on-premises SharePoint. Identity is mapped from the Office365 user to the on-premises user.

This means that in both scenarios there is correct data security trimming.

Beyond the Basics

Design considerations

What did Microsoft consider when designing the Office365 service:

• Crawl versus Query – Chosen to go the query root as the crawl infrastructure within Office365 was limiting. Also hundreds of thousands of tenants need to have a consistent performance maintained. Query helps to provide the best and most consistent user experience.
• UI Presentation – Users felt it was really important to see all the results on the same page. They didn’t want to have to switch between different pages. (I’m not sure I agree with this UX being the most optimum, users are used to choosing the ‘type’ of data they want on Bing/Google eg. Images/Shopping etc)
• Relevance and clicks – Search learns over time. Search watches which results are clicked for specific queries and adjusts over time.
• Background compatibility – Should provide other MOSS and SP2010 on-premises hybrid. This was not possible due to some significant infrastructure and services which would have required significant investment in extending the previous SharePoint versions. One major element was the challenge of identity mapping.

Alternative Hybrid User Experience

The demonstration showed the example of a glossary. This is stored in a SharePoint list on-premises. It is very useful information, but is not something users query for all the time. From on-premises the demo pulls ‘people’ results from Office365 via a dedicated search vertical.

To get the on-premises ‘people’ page to use the Office365 profiles it is as simple as pointing the web part settings to use the remote result source. This means when the users click the people results it goes to the Office365 profile page. Question is what happens to the links to people in normal item results?

Hit highlighting works.

Question about multiple people stores. Answer was the suggested best practice is to host in just one location.

Result Click-Through: Options

For users on-premises accessing on-premises links are fine. Once the user is on the Office365 search results the results now have internal only urls. The click through doesn’t route through the reverse proxy anymore. So users must have that external access to the internal system.

One solution is to have a VPN or Direct Access or leveraging the reverse proxy for url re-writing.

Microsoft recommend using VPN or Direct Access as it is easier to maintain over time.

The WOPI previews are operating where-ever the content is being served from.

Share and Enjoy