-
Updating and Migrating Confluence
I started hosting my wiki on Azure way back in 2016. The other day I received a message that I was not using managed disks and I should upgrade. So I pushed the upgrade button and kicked off a two day outage which resulted in me pulling my hair out, migrating, looking at logs, and…
-
Boilerplate Raspbian Setup
I tend to flash my raspberry pies a lot. Here are the steps I follow to get to my working state. Flash with latest raspbian image https://www.raspberrypi.org/downloads/raspbian/ Add an empty file named ssh to the boot folder Add an empty file called wpa_supplicant.conf Configure the wireless by creating a file called wpa_supplicant.conf country=US ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev update_config=1 network={…
-
More Time Doesn’t Always Help
Continuing on with Keras, there is a simple IMDB data set with movie ratings. Building a quick model and training over a few epochs shows the training accuracy goes up to the high 90%. However, the validation accuracy hovers around 87%. The model is over-fitting the data somehow. No matter how many more epochs I…
-
Surprising Results with Python Thumbnails
I’ve been working with images recently and am using a technique for a hashing algorithm. This technique resized the image to a 9×8 gray scale image. I found five different ways to do it with a few surprising results: cv2 pillow skimage two custom gray scale functions The Code The first part of the code…
-
Checking out TSLearn
I’m toying around with my new dashcam videos and thought I would try to build a neural network. I found Adam Geitgey’s article really interesting. My setup Surface Book Graphics Card GeForce 900M Series (Notebooks) GeForce 940M (1 GB) 5.0 Compute Capability Windows 10 x86_64 Python 2.7.14 Anaconda 5.1 CUDA 9.0 cuDNN v7.1.2 (Mar 21, 2018),…
-
Parse Azure Deployment Status
I’ve been making a lot of deployments that take a while to finish. There is an api call to fetch the status of the operations. azure group deployment operation list –name deploymentname –resource-group rgname The next step was to parse the json response. azure group deployment operation list –name deploymentname –resource-group rgname | python parse.py…
-
How Long to Deploy a Secure HDInsight Cluster
It takes a while to build a secure HDInsight cluster. In my case it was about an hour from start to finish. About half of the work was setting up the VNet and configuring the Domain Controllers. The HDInsight cluster took about 30 mins which his about normal. Some steps were dependent on other steps…
-
How I’ve deployed R Models with Microsoft R Server
Many organizations are benefiting from advanced analytics and machine learning. I work with a lot of teams who create amazing data analytics models and who need to then deploy them in a way to be useful. This is what I’ve seen and one way to deploy models. Generally deploying models run into two major obstacles…
-
How I fetched data from Hive in R with HDInsight
I have an HDInsight Spark cluster and wanted to fetch some data out of a Hive table and play around with it in R. I’m using Microsoft R. Here is how I did it.
-
Here are the differences between HDInsight and Hortonworks HDP
I get asked a lot about the differences between Microsoft HDInsight and Hortonworks HDP. Turns out there tight coordinated development efforts between the two companies. I was surprised to see how much closer the two products are now today compared to a year ago. Here is what I was able to find. Lag in…