Cosmos DB Failed to read item

Interesting one today – standing up a Cosmos DB to record the output of a CycleCloud job run which happened to be written in C++ and started getting “Failed to read item”. Data Explorer stopped showing the results from the item when browsing.

Issue was that our new id had been delimited with slashes and Cosmos DB didn’t like it. If you get “Failed to read item” when clicking through then you might have a character in your document Id that Cosmos doesn’t like.

https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.documents.resource.id?view=azure-dotnet

 

Azure, REST and some great community

There are some awesome folks out there who share their hard efforts so the rest of us can have an easier job. A few of these that have been really useful sit around work against the REST APIs of key Azure services.

My days of day in day out development are over so I find a lot of my automation “glue” mashing up deployments relies on PowerShell with the odd bit of CLI. Most is a little bit of scaffolding to deploy ARM templates but occasionally a requirement to work with the data plane of a resource appears and I have to resort to manual config.

ARM Template support for configuring resources is always improving but due to timing this isn’t always possible. Sometimes it is really helpful to understand what is going on, and sometimes the only option is REST.

For the latter I thoroughly recommend POSTMAN if you need to interact, though Azure is also improving native API exploring support. I discovered POSTMAN through an azure Friday video with Steven Lindsay who has some really really useful modules on GitHub. This is really helpful for CosmosDB (Documentdb as it was) and really helped me debug some Gremlin issues.

Next is the PowerShell module for CosmosDB which sits over REST and as well as being an awesome example of the kind is also a really helpful module for checking interactions with CosmosDB.

And finally an excellent post by Michał Pawlikowski on connecting to Azure Data Lake Storage with REST API with PowerShell, showing awesome detective work and publishing a bunch of really useful cmdlets.

 

 

Note to Self; Hyperfish and Kubernetes

Kubernetes and AKS in particular is becoming more and more important to us at work. In our experimental facility we have to stand up varying compute platforms; my main project is examining a specific workload on HPC and part of it needs Kubernetes to support some supporting work.

Then I stumbled across a blog by Chris Johnson . I’ve met Chris (officially a “good guy”) exactly twice in Person; once in 2010 in Berlin at an Ignite Session (when Ignite was a smaller scale effort) for SharePoint 2010 where he presented a session on Microsoft Certified Master, and secondly at Ignite in Orlando last year when I made a point of catching him before he presented a session of the Microsoft Cloud Show with Andrew Connell (also officially a “good guy”) and Julia White (yes, that Julia White).

Anyway, this is one of those posts which is as much for my benefit as yours!

Azure DevOps Rebranding and Git

Working in the Microsoft cloud ecosystem (ok, Azure) and working for a Microsoft Partner steers me heavily towards the tools that the vendor provides. This works on a number of levels; mainly around depth of knowledge and personally this means getting ready for the next exam.

For code and script storage this means Azure DevOps and GitHub, the choice has got harder lately due to the tweak to the “free” tier on GitHub and private repos but we all love Azure DevOps because of pipelines and all the other stuff, even though my primary day to day use is as a Git Repo.

Of course I’ve been using Visual Studio for years and the online version for as long as it exists. The rebrand to Azure DevOps also brought a new url option going from <org>.visualstudio.com to dev.azure.com/<org> and the latter has created some new joy. I really recommend Multi-Factor authentication and love using the latest and greatest tech from Microsoft including their security features as it’s about the only way to keep up with the threats we face out there on the internet.

Of course it comes back to bite you from time to time and this morning has been a classic case. The current Git for Windows Release is 2.21.0 but a key component for me as a multi-factor protected user of Azure AD and Azure DevOps is the Git Credential Manager for Windows and there are a bunch of fixes relating to the new dev.azure.com url in version 1.19. Git for Windows 2.21.0 unfortunately includes Git Credential Manager for Windows 1.18.4.0 so you’ll need to install in strict order to get this the correct way round.

My symptoms included the following:

  • No prompt for credentials when cloning my repo, just a couple of http errors then a prompt for a password.
  • No prompt for credentials even though I had removed the pat tokens and emptied Windows Credential Manager.
  • Errors thrown at the Git level (I tend to live in VS Code or Visual Studio).

Musings on Azure CycleCloud

One of the (many) great aspects of my current role working in the Innovations area of a UK Bank is a relentless introduction to new features in Microsoft Azure. At my stage with Azure in practice and exams it is usually a new feature or behaviour that has dropped as part of a generation 2 (E.g. Storage vs Data Lake) or evolution of features or more subtly a change to the defaults of a combination (e.g. Automation and Desired State Configuration Extension). Then there are the “never heard of it” moments when a term gets mentioned and I rattle straight to a search engine.

One of these a few months back was Azure Cyclecloud, one of our projects involved input from Microsoft and their HPC specialist proposed it as a key component of the platform being evaluated. In our case it is acting as an orchestrator / scheduler and keeping tabs on a handful of low priority virtual machine scale sets.

I’ve not had any direct exposure to HPC beyond awareness due to Microsoft architectural exams I’ve done in the past for on-premises Windows, and latterly Azure cloud. The good news is within parameters that the news is good and Azure CycleCloud appears straightforward and being predominantly IaaS based is fairly easy to secure within our patterns. My thoughts so far are:

  • The web admin interface is fairly sensitive to environment – I’ve lost about a day to Internet Explorer (doesn’t work) and the reverse proxy on our firewall appliances mangling page scripts.
  • The manual install is straightforward and reliable in my limited experience – we have a vnet model that it sits in quite nicely and the documentation is good on required ports and cluster communications.
  • Azure Cyclecloud being HPC and batch etc comes from open source land, so lots of command line and Linux – quite ironic that my career includes so many loops (my first job at an accountants in the 1980s included being the guy who wrote sql reports using vi on a unix practice management system).
  • Following on from the previous point, Azure Cyclecloud integrates with Active Directory and therefore has it’s own RBAC model – very important to understand if you are trying to secure it.
  • I have a few concerns about the quickstart deploy, mainly due to the public ip address bound to a server but that probably reflects our use cases and my background. (Googling “cyclecloud initial setup” reinforces this concern as a number of servers in initial setup pop up).
  • The cloud account relies on a fairly big service principal so it’s important to keep on top of that bearing in mind the last two points.
  • About 50% of the time I get the name wrong and call it Azure CloudCycle. This hit rate is slowly improving.

Microsoft Certified Professional Exam Status

Where do I start? Migrating this blog over the weekend has led to a bit of a review and the realisation that a lot of the blog posts relate to my journey preparing for and sitting (and generally passing!) Microsoft Certified Professional (MCP) exams.

The last exam related post on this blog is Passed 70-631 WSS Configuring Today which was posted just under 10 years ago – yikes. I’m delighted to see that posts in the meantime related to Motorcycling and Off Road Skills so that would indicate some wider interests other than work.

As I write I am two weeks from passing my most recent Microsoft Exam Continue reading “Microsoft Certified Professional Exam Status”

SharePoint 2013 post CU says Server Error: http://go.microsoft.com/fwlink?LinkID=177673

The situation was that I thought I would bring my SharePoint 2013 up to date with the latest cumulative update, at the time of writing December 2013 as we are still waiting for the issue with SP1 to be remedied.

So the cumulative update ran through fine and apart from a few complaints in the upgrade file to do with PowerPivot, all was well. So I went in to a demo site and got an empty screen with the following: Server Error: http://go.microsoft.com/fwlink?LinkID=177673. When you click on that link it takes you to an update deployment page for SharePoint Foundation 2010.

So I checked central administration upgrade status – nothing. And checked the database upgrade status, all fine. So I went with my cavalier-lazy and ran the SharePoint 2013 Products Configuration Wizard. After the usual multi-stage process it completed with no complaints. Checked my site, no joy – same result.

So I had a hunt around and fired up the SharePoint 2013 Management Shell and Test-SPContentDatabase on the content database for my demo site and nothing. So I ran it as administrator and again got nothing. So I tried Upgrade-SPContentDatabase and it helpfully told me that my content database didn’t need upgrading.

So I had another hunt around and tried psconfig -cmd upgrade -inplace b2b -wait -force and again got no errors. So I admitted defeat and started hunting through my ULS logs. This threw up interesting errors, the start of which was:

04/21/2014 10:55:41.99  w3wp.exe (0x3208)                        0x237C SharePoint Foundation          Database                       880i High     System.Data.SqlClient.SqlException (0x80131904): CREATE TABLE permission denied in database ‘

So again I took the lazy approach and fired up SQL Trace on my database and narrowed down the filter to have a Database Name like the content database and ran it up, and assuming it was psconfig related ran that again. The command ran through with no errors reported to the command line and when I stopped the trace and had a look I didn’t see anything. And nothing in the ULS log. So I waited. And the same entry appeared again – don’t you love asynchronous stuff?

So I was a bit more patient this time – I restarted the trace, and ran psconfig, and waited until the error appeared in ULS. Then stopped the trace and went looking. And there it was, a big long SQL statement running under an account that didn’t have sufficient priviledge to create the table. So being a hacker I gave that account dbo priviledge – ran psconfig and after a suitable pause my site is running again.

The interesting bit ? The identity wasn’t the application pool identity of the web application, it wasn’t the identity of the timer service (Ok I know it wouldn’t be, owstimer.exe wasn’t owning up in ULS). On my VM it was the identity of the portal web application / community sites. I’m nonplussed but will put it on my “to learn” list.