Skip Ribbon Commands
Skip to main content

Liam Cleary [SharePoint MVP]

:

Home
November 06
New Blog Site

​Over the next few weeks and months, I am going to move my blog site from here to a new site running in the Azure Cloud. I will be maintaining two URLs for a short while, until I migrate everything:

http://blog.helloitsliam.com - Current

https://www.helloitsliam.com - New

Once everything is migrated over I will point the http://blog.helloitsliam.com URL to the new site. So update your bookmarks and feeds to use the new URL.

For now I will post ALL new stuff to https://www.helloitsliam.com.

Thanks fore you're patience as I migrate everything. 

November 05
SharePoint Online and Data Loss Protection (DLP)

So as you all should know by now, Data Loss Protection (DLP) was introduced into SharePoint Online earlier this year. This is really just an extension of the eDiscovery process in SharePoint, it is really used for checking the content you have in the site already. As I have spoken with other SharePoint people in the community and clients I have been surprised that not many people have used or even seen it.

To learn more about DLP as a whole, then review this video and details from TechNet: http://technet.microsoft.com/library/jj150527%28v=exchg.150%29

So to hopefully help with this, let's take a look at it. Firstly we are talking SharePoint Online, so you will first need to have access to your tenant, and a site/site collection based on the "eDiscovery" template.

To make sure it should be look like this:

To start, we need to create a new case. To do this simply click the "Create New Case" button.

I have set the properties as shown below:

 

You're case site should then be created and you should be taken to that site and should be ready for you to use.

So the first part of the DLP "search" process is to perform a "Search and Export" query. I named mine "Credit Card PII" as shown below.

As you can see you are able to add some free-text query, dates, and names or email addresses as well as set a scope for the results. At the bottom of the screen you get results back from Exchange or SharePoint.

So the plan here is to find any content that contain credit card numbers, so first off make sure you actually have content in the site that contains credit cards or this won't work. Then we will start to add a query to retrieve the content.

Let's start by understanding the syntax that should be used. The queries should be set in the following format:

SensitiveType:"{Type}|{Count Range}|{Confidence Range}"

The "SensitiveType" is required whereas the ranges are optional. The syntax can also be added to using regular search syntax.

An example of this combination could be something like this that finds all documents that contain 2 to 15 "Credit Card Numbers", within all file types except "PDF" files.

SensitiveType:"Credit Card Number| 2..15" AND NOT FileExtension:PDF

If you want to learn more about using search query syntax in eDiscovery you can visit the following documentation.

http://blogs.technet.com/b/quentin/archive/2014/07/30/using-search-properties-and-operators-with-ediscovery.aspx

So our example will be something simple to just get anything with credit card numbers in. We would use the following:

SensitiveType:"Credit Card Number"

We could have chosen something else instead of "Credit Card Number", to see the fully supported list you see the table below, or visit the TechNet page.

http://technet.microsoft.com/library/jj150541%28v=exchg.150%29.aspx

Information type name

Primary region

Category

ABA Routing Number

United States

finance

Australia Bank Account Number

Australia

finance

Australia Driver's License Number

Australia

PII

Australia Medical Account Number

Australia

health

Australia Passport Number

Australia

PII

Australia Tax File Number

Australia

finance

Canada Bank Account Number

Canada

finance

Canada Driver's License Number

Canada

PII

Canada Health Service Number

Canada

health

Canada Passport Number

Canada

PII

Canada Personal Health Identification Number (PHIN)

Canada

health

Canada Social Insurance Number

Canada

PII

Credit Card Number

All

finance

Drug Enforcement Agency (DEA) Number

United States

PII

EU Debit Card Number

European Union

finance

Finland National ID1

Finland

PII

France Driver's License Number

France

PII

France National ID Card (CNI)

France

PII

France Passport Number

France

PII

France Social Security Number (INSEE)

France

PII

German Driver's License Number

Germany

PII

German Passport Number

Germany

PII

International Banking Account Number (IBAN)

All

finance

IP Address

All

PII

Israel Bank Account Number

Israel

finance

Israel National ID

Israel

PII

Italy Driver's License Number

Italy

PII

Japan Bank Account Number

Japan

finance

Japan Driver's License Number

Japan

PII

Japan Passport Number

Japan

PII

Japan Resident Registration Number

Japan

PII

Japan Social Insurance Number (SIN)

Japan

PII

New Zealand Ministry of Health Number

New Zealand

health

Saudi Arabia National ID

Saudi Arabia

PII

Poland National ID (PESEL)1

Poland

PII

Poland Identity Card1

Poland

PII

Poland Passport1

Poland

PII

Spain Social Security Number (SSN)

Spain

PII

Sweden National ID

Sweden

PII

Sweden Passport Number

Sweden

PII

SWIFT Code

All

finance

Taiwan National ID1

Taiwan

PII

U.K. Driver's License Number

United Kingdom

PII

U.K. Electoral Roll Number

United Kingdom

PII

U.K. National Health Service Number

United Kingdom

health

U.K. National Insurance Number (NINO)

United Kingdom

health

U.S. / U.K. Passport Number

United States and United Kingdom

PII

U.S. Bank Account Number

United States

finance

U.S. Driver's License Number

United States

PII

U.S. Individual Taxpayer Identification Number (ITIN)

United States

finance

U.S. Social Security Number (SSN)

United States

health

 

So we can use any of the syntax above to search for any type of PII data. Next I am going to set the source to my team site, currently have a file stored there with the content in. To do this I click the "Modify Query Scope"

Next I will select the "Add Location" and past the URL of my Team Site and check it.

Now we should have a query form completed as shown below.

Adding a location will change the underlying the search query to be the following:

Now if we run the search we should get the results back as expected, obviously having to wait for Office 365 to perform a search crawl, which could take a while J

Of course the next step is to export the results or just to save it and perform the required hold and remedial work to fix this issue. This is the first implementation within the Office 365 platform, with great plans I am sure for future updates.

October 22
10 Ways to a more Secure SharePoint [Infographic]

 

Crafted by: Sharegate The SIMPLEST SharePoint Management tool suite.

September 08
SharePoint Alerts via SMS? @IFTTT

So I like to play with non-SharePoint technology such as Lego Mindstorms, Raspberry PI and of course Linux along with hacker equipment. One thing I have been meaning to look into is a service called "IFTTT" which stands for "If this then that". It is great service that allows you to create combinations of actions and events that ca be used with other technologies. For example if you have not seen the "Littlebits" components then go there now. The "Littlebits" components are for electronics that can be connected to the "IFTTT" services to add "Cloud Based" function easily.

As I was playing with this the thought came to mind "Could I utilize this to send me alerts from SharePoint?", so I am clear in the real world I would use something else if I wanted to achieve this, but this service works fantastically well for other things such as working with an Arduino, Raspberry PI or the "Littlebits" Cloud module.

Of course I was going to use it with SharePoint to test anyway J

So first things first you have to set the trigger you would like to use. I selected the "Email" as that is how SharePoint sends alerts out.

Now I get to choose how to interest with it. I chose the first option.

Now I complete the "Trigger" and move to the next step.

Now I get to determine the "That" clause.

For this I am going to choose the "SMS" option.

I simply activate the "SMS" Channel.

Simply complete your phone number and then get the pin sent.

Once you have validated the pin you are told it is active and then you can complete the "SMS" channel activation.

Now I choose to the action which for the "SMS" channel is only to "Send me an SMS".

You are then able to construct the "SMS" message as needed.

I constructed mine as shown below:

I simply created the action and now get to confirm the whole "Recipe" as it's called.

My "Recipe" is now enabled and should be working.

To test it I simply sent an email to the trigger@ifttt.com email address from my designated account and hey presto it pops up on the phone.

This a great "free" way of using any service that converts emails to "SMS". Even better is that you can use this for all kinds of things. You can even use this same approach and append a "Tag" to perform various other functions not just "SMS" messages. I used SharePoint as the example but in reality I would use if for other things such as when I power on my Raspberry PI, it sends me an "SMS" telling me it's IP Address or even better if I am leaving something to sniff the network and send packets I could use this service too, utilizing the email recipe or others.

 

 

September 03
End of the summer, Start of Conference Season Again & the Community

So like most of you, the summer holiday / vacation season is over and work and school resumes as if it has never gone away. For me it has been a busy summer, everything from having parents and other family in town from the UK, to running a Youth Conference with some great people, then owning and running a Scout Camp with great leaders for a week, as well as trying to spend time with family and of course still work. All in all it has been a busy summer and the autumn / fall doesn't look any easier either. In the next week I will be on the road hitting the following conferences:

SharePointalooza, Missouri

IT / Dev Connections, Las Vegas

SPTechCon, Boston

SharePoint Fest, Denver

SharePoint Summit, Vancouver

MVP Summit, Seattle

SharePoint Saturday Twin Cities, Minneapolis

SharePoint Fest, Chicago

 

I ended up turning down the offer to speak at a Security Conference called Derbycon which I was massively gutted about. That conference is full of very cool "hacker" type people and would have been great to speak and attend, but having to tell my dear wife the 10 days plus I am away now needs to be extended further did not sound like something I really wanted to do, plus believe it or not I actually miss my family when on the road J

So what does the next few months hold for me? Lots of work as normal and travelling to share the good word around the US. I already have conferences stateside and international for next year I am scheduling now, so I am sure you will see me around somewhere.

As I was thinking about the next few months, I am reminded how busy we all are in the SharePoint World, busy working, busy sharing, busy travelling and busy going to conferences all over the place just to be with each other and share what we have learned. I am, grateful to a community that helps when it is needed, is more than willing to be there when struggling to resolve issues, but more importantly to create great friendships built on our common struggles with delivering the right solution. Sad to think that with the technology changing is it always does that really in my personal opinion the community will change, people will drift to other things and the community as it is today, will become a distance memory of how things used to be.

Change is good, and the more I see how things are changing where things are moving too, all is good. We will always have SharePoint work to do, whatever it ends up being called in the future, we will always have a need to teach and share what we have learned and always have a need to create friendships with those around us built on the common technology we work with.

I look forward to the next iteration of things to come, spending time with you guys in the unnamed community in whatever role I can play. Of course with my kids getting older, more activities and of course to think that my daughter is taking driver's education this year makes me realize that time is very precious and seems to run away from me with no thought at all.

On that note, hopefully see some of you out on the road over the next few months, spreading the good word that right now is still "SharePoint" J

August 26
SharePoint 2013 Workflow Manager Woes

While building some recent SharePoint 2013 Servers, I hit a few issues relating to the Workflow Manager Service installation. The Servers are running Server 2012 R2 and as such no matter what order I selected the components for installation it would always fail and some pieces would install and others would not. After playing for a while, I decided that the only way of doing this would be to perform a manual installation and not use the "wpilauncher". This is done by using the same tool but from a command line instead.

The order is very simple you need to download in this order:

  1. Service Bus 1.0
  2. Service Bus 1.0 Cumulative Update (KB2799752)
    1. Download here: http://www.microsoft.com/en-us/download/details.aspx?id=36794
  3. Workflow Client
  4. Workflow Manager Refresh

The command line approach is very simple, it uses the following syntax:

Webpicmd /offline /Products:"Product Name" /Path:"Path Location"

To download the components above you can use the following commands, I am using the following location: "G:\Components\Workflow\" to download into.

Webpicmd /offline /Products:ServiceBus /Path: G:\Components\Workflow\ServiceBus
Webpicmd /offline /Products:WorkflowClient /Path: G:\Components\Workflow\Client
Webpicmd /offline /Products:WorkflowManagerRefresh /Path: G:\Components\Workflow\Manager

Make sure you download the Service Bus 1.0 Cumulative Update using the link above and place in the same location. My folder structure looks like this:

Now to perform the installation you need to use the following command:

WebpiCmd.exe /Install /Products:Product to Install /XML:XML Definition Location /AcceptEula /SuppressPostFinish

This syntax will allow each component to be installed using the command line in the correct order. The following are sample commands you can run:

WebpiCmd.exe /Install /Products:ServiceBus /XML: G:\Components\Workflow\ServiceBus\feeds\latest\webproductlist.xml /AcceptEula /SuppressPostFinish

 

G:\Components\Workflow\ServiceBus-KB2799752-x64-EN.exe

WebpiCmd.exe /Install /Products:WorkflowClient /XML: G:\Components\Workflow\Client\feeds\latest\webproductlist.xml /AcceptEula /SuppressPostFinish

 

WebpiCmd.exe /Install /Products:WorkflowManagerRefresh /XML: G:\Components\Workflow\Manager\feeds\latest\webproductlist.xml /AcceptEula /SuppressPostFinish

 

Once you have ran the commands, everything will be installed and working. You can then either use the standard Application Wizard to set it up, or PowerShell. I would encourage you to use PowerShell, which is much easier and needed if you want to repeat the process easily.

 

PowerShell: If you wish to use PowerShell instead of "old skool" command line, then simply use this syntax:

 

Start-Process -FilePath "Path to WebpiCmd" -ArgumentList "/Install /Products:"Product to Install" /XML:"XML Definition Location" /AcceptEula /SuppressPostFinish" -Wait –PassThru

 

An example would be:

 

Start-Process -FilePath "C:\Program Files\Microsoft\Web Platform Installer\WebpiCmd.exe" -ArgumentList "/Install /Products:ServiceBus /XML: G:\Components\Workflow\ServiceBus\feeds\latest\webproductlist.xml /AcceptEula /SuppressPostFinish" -Wait –PassThru

 

 

Now outside of this once all this was installed I had various issues connecting a second server to the Workflow Farm. You can see this when you run the wizard for the new servers you wish to add and upon validation you see port blocking errors similar to this:

 

 

The first places to check would be the Windows Firewall rules to make sure the rules that were supposed to be created actually got created. They should be the following:

 

 

Even if the Windows Firewall is not running (which of course won't be the issue) you should have these rules generated and added. Next check that the services on the primary server (first one created) are actually running. I found that the "Service Bus Message Broker" was not running at all. Resolving this is needed either way. Check the following services are all running:

 

  1. Service Bus Gateway
  2. Service Bus Message Broker
  3. Windows Fabric Host Service

 

If any of these are not running, start them, you may need to reboot the server to have them start cleanly. So if that doesn't fix it then you need to look at traffic between the servers you are trying to connect to. For my instance I was working with, it was firewalls blocking the communication between the two servers. The following rules needed to be added to allow the traffic:

 

  1. Allow Port 9355 HTTPS
  2. Allow Port 9354 TCP
  3. Allow Port 9356 TCP
  4. Allow Ports 900 to 9004 TCP

 

Once these changes were made, the wizard re-validation process still failed. So far I had checked all the network connectivity, services and what I thought was everything. So what I decided to do was run the wizard for removing the server from a Workflow Farm. I was quite amazed that it offered me the ability to do that, so at some point it had connected itself only for the Service Bus components to the farm.

 

 

After running that wizard, it completed successfully saying it had removed the current server from the Service Bus Farm. Re-running the "Join to Farm" wizard then successfully validate the connections as needed.

 

 

The wizard then ran and failed again, based on the "Service Bus Message Broker" service not being able to start. I still don't have it working 100% yet due to the service just not starting at all, but I am now further along than I was a while ago. When I get it working 100% I will post an update.

August 19
SharePoint Security Roundtables

Just wanted to say a big "THANK YOU" to all those that attended the couple of SharePoint Security Roundtables I did in McLean, VA and then SanFrancisco, CA. It was a great time to talk about some of the issues we face as businesses and organizations in securing our content. Looking at the things we need to consider, and then how we can govern it a little bit better. 

We still have two more planned being presented by my colleague Doug Hemminger:

http://sharepoint.protiviti.com/events/SharePointRoundtables/Pages/default.aspx

If you are in Chicago or Minneapolis then sign and let’s have a great conversation about Security and SharePoint.​

July 01
Annual Collaboration Security Survey
A year ago the guys over at Cipherpoint conducted a survey of collaboration systems. This included SharePoint, file servers, Office 365, and SharePoint Online. The purpose of this was to explore security and compliance issues related to the use of these platforms​. If you have not seen this then head over to the link below:

http://www.cipherpoint.com/company/sharepoint-help/cipherpoint-2013-state-of-collaboration-security-report/

The great news is that they are doing this again this year and need YOU to go and complete the survey. Use the link below to complete it.

http://cipherpoint.actonsoftware.com/acton/form/5891/0015:d-0001/0/index.htm

I encourage you to do this, so that we can then look back as an industry at what are the security holes and issues that we all face with collaboration platforms.
June 02
SharePoint 2013 Search Crawl Weirdness

Today I found a little weird thing. A client SharePoint 2013 environment would not crawl at all. It caused the following standard error that I am sure you have all seen a million times.

"Item not crawled due to one of the following reasons: Preventive crawl rule; Specified content source hops/depth exceeded; URL has query string parameter; Required protocol handler not found; Preventive robots directive. (This item was deleted because it was excluded by a crawl rule)"

I tried everything I could think of, deleting and re-creating result sources, crawling one page, crawling not as SharePoint, removing the "robots.txt" files to name a few things. I then resorted to trawling through the "web.config" and found the following entries:

<customHeaders>   

<add name="X-Content-Type-Options" value="nosniff" />

 <add name="X-MS-InvokeApp" value="1; RequireReadOnly" />

</customHeaders>


HTTP Header <add name="X-Content-Type-Options" value="nosniff" />

Each type of file delivered from the web server has an associated MIME type (also called a "content-type and not a SharePoint Content Type either") that describes the nature of the content (e.g. image, text, application, etc.). For compatibility reasons, Internet Explorer has a MIME-sniffing feature that will attempt to determine the content-type for each downloaded resource. In some cases, Internet Explorer reports a MIME type different than the type specified by the web server. For instance, if Internet Explorer finds HTML content in a file delivered with the HTTP response header Content-Type: text/plain, IE determines that the content should be rendered as HTML. Because of the number of legacy servers on the web (e.g. those that serve all files as text/plain) MIME-sniffing is an important compatibility feature.

Unfortunately, MIME-sniffing also can lead to security problems for servers hosting untrusted content. Consider, for instance, the case of a picture-sharing web service which hosts pictures uploaded by anonymous users. An attacker could upload a specially crafted JPEG file that contained script content, and then send a link to the file to unsuspecting victims. When the victims visited the server, the malicious file would be downloaded, the script would be detected, and it would run in the context of the picture-sharing site. This script could then steal the victim's cookies, generate a phony page, etc.


HTTP Header <add name="X-MS-InvokeApp" value="1; RequireReadOnly" />

The "DirectInvoke" feature in Internet Explorer enables applications to register their MIME Types for direct invocation from a URL. When Internet Explorer encounters a file type that it doesn't handle natively, it can use a handler application, rather than downloading the file. Using "DirectInvoke", handler applications have control over how their files are downloaded and enables smart techniques specific to the application's requirements. Microsoft Office maintains a local cache of documents on a user's machine. When a user attempts to download an Office document in Windows Internet Explorer, "DirectInvoke" calls Office using the target URL, rather than downloading the file. Office checks its cache and only downloads the file if it isn't already in the cache. This behavior can provide significant bandwidth savings, especially when handling large, media-rich documents.

When the "web.config" is set to "1;RequireReadOnly", then the server is requesting that a "DirectInvoke" configured application be used and requires that the file opens in read-only mode.


For this environment I removed the two lines and the crawl worked. I need to do some further investigation but right now this worked perfectly. The security and functionality impact for the "default" zone crawl in my mind are negligible, for the external side I would not remove these at all. I will post updates once I have more detail but for now, this has done the trick. More to come :-)

 

May 10
European SharePoint Conference 2014

Well what a great place to have a SharePoint Conference, Barcelona!! It has been a few years since I have visited now that I live in the US. After flights to the UK, then onto Spain, I arrived very tired but ready to go for it. The Hotel is fantastic, Hotel Rey Juan Carlos, set within great grounds and across the street from the conference center.

The conference has been great so far, fantastic keynotes and sessions covering all types of content. I did my "Hacking" SharePoint session, the one I thought I would present a few times and then never update or be asked to do it again. However as you may have seen in the news, Security and Hacking is becoming more important in every facet of our life that I keep updating the presentation!! Having presented it lots and lots, I look back and realize it has been different every time. I was excited to present that here. I want to thanks those that came and filled my room for the session, and just let you all know that the guy on the webcam woke up J

NOTE: Blurred on Purpose, it is not your eyes!! You needed to be in the session to see it properly J

 

 

For those that were not in the session, this is how I started the session, explaining that we still don't secure things the way it should be done. Simple things like someone's (no idea who's it is) webcam at home. Accessing this one was as simple as first finding it, then looking up the default username and password for the specific manufacturer and model. IN this example it was a FOXCOM FI8910W, which a quick search in Google points me to the credentials I needed.

 

This for me is the problem, why was the password not added, or username changed. This is really the premise of the session. I often use the analogy of buying "flat-pack" furniture, something you would buy from Ikea for example. We all know that it contains instructions, but we all seem to inherently know how it is supposed to go together though we may have never done it before. So we put the furniture together and find something is not quite right. We seem to do the same with SharePoint installations. It is so easy to set it up incorrectly, but we do it without following guidance just as with building the furniture. The documentation is better now than it has ever been, we need to follow this so that we make our environment as "least privilege" as possible and published securely.

We also need to now think like a potential hacker, testing our platforms whether internal or external. We need to also keep up to speed on cyber-attacks and specifically what Microsoft is doing around this. Simply visit http://www.microsoft.com/security/cybersecurity/#!Overview to see the cybersecurity whitepapers and approaches. Of course you can also visit https://technet.microsoft.com/en-us/security/bulletin to see the latest security bulletins.

 

 

 

All in all hopefully the message that you are getting is to check your SharePoint environment, make sure it is setup, configured and secured correctly. Penetration test your environment to see what you can find. Below are some links to current blog posts, and there is a series I am still working on that will show you how to penetration test your environment as an example of what can be done.

SharePoint URL Endpoints (Use in Google)

http://blog.helloitsliam.com/Presentations/Urls.txt

 

Is Your SharePoint Secure – Part 1

http://blog.helloitsliam.com/Lists/Posts/Post.aspx?ID=100

 

Is Your SharePoint Secure – Part 2

http://blog.helloitsliam.com/Lists/Posts/Post.aspx?ID=101

 

Is Your SharePoint Secure – Part 3

http://blog.helloitsliam.com/Lists/Posts/Post.aspx?ID=103

 

Is Your SharePoint Secure – Part 4

http://blog.helloitsliam.com/Lists/Posts/Post.aspx?ID=105

 

Hacking versus Misconfiguration

http://blog.helloitsliam.com/Lists/Posts/Post.aspx?ID=115

 

Is Your SharePoint Vulnerable?

http://blog.helloitsliam.com/Lists/Posts/Post.aspx?ID=116

 

Last but not least I demonstrated quickly the use of a tool called Maltego that allows for gathering of information from multiple sources, to build a picture of people, devices or content.

 

NOTE: Blurred on Purpose, it is not your eyes!! You needed to be in the session to see it properly J

 

This is a great tool, requires a bunch of learning, but is very useful for gathering information about details you may find from trawling through SharePoint sites.

As per my last final thoughts in the slides following these steps:

  1. Pen test your SharePoint Site – plenty of tools out there for this
    1. Internal - Choice
    2. External – No Choice
  2. Ensure Latest Patches
    1. My personal rule be two CU's behind, unless you need the CU for bug
  3. Users will find a way of getting into content, just as they did with file shares
  4. Hackers will always try to circumvent security
  5. Learn how to hack!! Just Kidding 
    1. At least how to protect against the hack
  6. Make Security Top Priority
  7. Learn how to publish SharePoint correctly and securely

 

The pen testing series will continue soon, look for a new post soon J

1 - 10Next

​ ​​​​
Looking Cool!!