🚀 Automagical Fairs and Shouts to Fellow Pirats using Service Slave Workers 📣

When plundering and sailing keeping track of pesky DM’s is no job for the captain or crew. This’ why we summoned the services of Power Automate and Power Apps to do our bidding: Enabling our ship to run smoothly and commands being forwarded at the correct time to the correct people.

It all starts with the Consumer of Data “Power Automate” who collects data from all our different intelligence points. Working in the shadows, this ghost can be attached to any workflow. Currently, since it thrives on human flesh for its operations, we started by attached to our crew log. Whenever Power Automate smells the fear of a new deck hand, laddie or lassie onboard it goes to work!

On a new pirate record Power Automate does two things:

  1. Creates an Adaptive Card in teams and posted it to the ships group chat
  2. Sends a welcome e-mail to the fresh pirate, explaining how they need to obey or walk the plank (all pretty standard pirate stuff).
Current Dataverse Tables the flow can write to and read from.
Power automate flows that creates and updates records, when records for pirates are updated we trigger a Teams Integration.

Making the Captain’s work Screen Agnostic

We further improve the Captain’s life with Power Apps connected to the Power Flows. The Power App sits on the users phone, implemented via Teams or as a standalone app, and enables the Captain to administrate the crew roster, so any stowaway can be added to the crew and receive the required e-mail package with plank rules and NDA’s. This mobile approach lets the captain be in full control even when venturing away from his cabin and the desktop computer therein.

Screen snippet showing Power Automate triggering Teams Adaptive Card on new shipmate record.

The convenience of having the Power Apps in teams means the captain can manipulate his Ship, The Raids and roster from any surface, be it mobile, desktop og even tablet!

Screen snippet showing the Power App integrated into a Teams. Notice the responsive nature of teams adapting to any viewport.

Using Power Apps means we can also simulate raids adding new test data on the fly, without it needing to come from our Power Automate flow.

User generated Raids in the raid table, created directly within Power App.

In Summary, for Sleepy Pirates:

  • The Power App enables our users to directly manipulate our data storage.
  • The Power Automate allow other sources and application to remotely access our data.

Capture new pirates

When we need new pirates on our crew we have an app where we can take their picture, and the app send the picture to a server that uses AI to piratefy your face.

The app is a Canvas App with a camera control on the first screen

When user click the camera control they navigate to the next screen where they will see their picture.

They can go back and take a new picture if they are not happy, or move on to “Piratefy me”.

If user click the “Pratefy me” button a power automate flow is triggered. We pass the image base64 value as a value and store the reply value in a variable.

The flow is “Me as a Pirate”

The trigger is a power apps trigger.

First we initiate reusable variables

We grab the image value from the Power Automate

Passing the image over to the CLIP service powered by Stable Diffusion at the Frogner/Oslo datacenter we annotate images to mostly correct the gender or non-gender of the person in the photo. This is our responsible way to ensure the pirate representation of the person is not turning the most beautiful guy and girl into something they definitely don’t identify with.

We parse the response we get

Next we pass in the captured photo, and overlay it with some serious pirate’y prompting to ensure our new crew mates meats the bar (and also the real bar when port-side).

We need to modify the image string to make it the right format for the Ai to handle.

We trigger a child flow (documented further down)

We gather the reponse and send it back to the Power App.

The flow triggered as a Child flow is called “get Crew Member Data – PirateGDP”

The core API in out solution is PirateGPT, a large language model fully capable assigning awesome pirate names and corresponding ships, with the identifiable traits of the pirate. Which we as a responsible HR ship keep fully secure and up to date as needed. When you have seven John’s on one ship, it’s quite important to distinguish them by traits such as red beard or cock farmer.

We PARSE the results we get for get the right format

Respond with the values to the “parent” flow

The Power App

Back in the Canvas App the user will see the newly created name and their picture Piratified:

It also navigate to the next screen where they will see the pirate name they are given and their picture Piratified.

If this isnt glossy and extreme business value, I don’t know what is?

Imagine this being a manager taking pictures of new employees. We all know that Gen Z are used to looking at themselves with prettifying filters. This way the old middle manager can take pictures of their new employees and let Open AI add the familiar filters so that when the Gen Z employee see their own new profile picture they will actually recognize themselves.

Badges

Glossy Pixels – because pirates are beautiful.

Go with the Flow – because it’s all flowing left and right

The Existential Risk – Who knew LLM could be used to generate schematized JSON data structures, and who knew all those crazy internet pics would be the basis for pirate faces of 2023?

ACDC Craftman – All API’s are tested and documented in a shared Postman collection, all is ran over https (not from the beginning) to avoid photo snooping, and API keys are used as the extra layer. No one wants to be the next TMZ victim!

Mates Upcoming Raid…

Eyyrr mates!!! 

Captain Black Bart, is planning a big raid, Oslo is the target and according to ChatGPT the loot potential is #awsome.

Invitation is sent to all chosen crewmates using top of the art “Power Automate flows”. 
Crewmates are free to Accept or Decline the raid invitation.   

When it’s time for the raid, we requires all crew members who have accepted the raid to check in, using their RFID card or phone. 

If the crewmate have accepted more than one raid, he/she can choose to check in using our Raid Planning App 

With the help of Midjourney AI, https://docs.midjourney.com/,  we generated the app design component by asking the AI to provide us witht the latest & greates piracy themes.  
 

The RFID data for all potential crewmates is stored in Azure AD using the “Postal Code” field, pretty “Nasty Hack” according to our “bosun”. 

This data is imported to Dataverse “AAD User” virtual table for our app. 

Let’s raid mates. 

Kraken’s Masterpiece

As we have mentioned before, our ship data comes from Kystverket. We expect high data quality and in the early days of pirating this was achieved by importing excel lists where the pirates had used much time to track down the ships in order to maintain accurate positions. Now this work poorly as the amount of ships has increased and the time used to maintain the lists resulted in ships sailing to the next city before we could even detect it. In other words, the ship had sailed. Therefore, we had to make a more efficient way to load the same data and keep it consistent with Kystverket. By using power automate we created a flow that contacts their api and fetches data from premade bounding boxes around the Norwegian coast. It deletes ships out of range and updates the location of those who remains. If new ships occur in our areas, they are created in Dynamics 365 and we will know!

By making the data collecting process more efficient for our pirates using data from other sources, we hereby claim the Go With The Flow and Dataminer badge!! Argh 🙂

New Pirate Registration – Flow and 3rd party data

Pirates and singers are alike. They need a great stage name for people to remember who they are. When a new pirate arrives at the harbor master looking for work, the first thing they need to do is find a good name.

Just imagine what EMINEM would be if his artist name was Marshall Bruce Mathers III !!

The Harbor Master enters the new pirate to the system, and lets the AI tool decide what type of name suits the new pirate the best.

On the save, a Power Automate fires off a querry to OpenAI via a custom connector with the querry “Give me a pirate name for *Full Name*”. The return is then entered on the contact card.

Flows just makes everyones day better!

Our Pirates are not happy that all the tasks they get assigned in F&O only are available there. Why should they need to log in to F&O to see them or get an email notification that they have been assigned a new task? We all know that there are more than enough emails going around, so let’s make it easier for everyone. This flow automatically includes the tasks that are assigned to you for Onboarding or Offboarding processes in F&O directly to you Microsoft To-Do list through Planner. Then they pop stright up in you list, and you can handle it from there. Pretty cool, right?

The Jackpot App :: Pirates fleet enlisting

The reason we are calling this the jackpot app because we believe that it’s hitting multiple score-points!

The idea of this app is to enlist pirates’ fleet without a single character punch!!! Aided and infused by AI Builder, external API’s and Power Automate Flows; this Power Apps canvas app is state of the art geekiness!

Although, registered vehicles are the last of our worries (as pirates), but hey we thought of doing it anyways 🙂 The app starts by taking a picture of the registration plate of the vehicle. the app will send the picture to the AI Builder through a power automate flow that process the image and read the registration plate and show it on the app for confirmation.

Once the you are satisfied with the result, you can check the vehicle against the national registry and search for more registration information. This indeed does check the car against the Norwegian car registry (Statens vegvesen) show if the vehicle is approved for going on the European roads (EU-godskjenning). Also least of our concerns!

Actual data from Statens Vegvesen
Crawling and datamining data from external sources

Now, after confirming and inserting car data into our system. We need to identify the vehicle type and compare with our collected data in our backend system. However, identify the care is a simple process that doesn’t need any text input from the user. One picture will do while the AI Builder will take care of identifying the type of the vehicle and insert it into our backend system.

Vehicle plate identification flow (calls another flow through HTTP)
AI Builder made as an endpoint (all AI Builder models and flow are on separate environment, as AI Builder
can’t be included in a solution for deployment)

We are using a mix of built-in and custom AI models to achieve efficiency and productivity in an innovative way.

AI Builder model

They see me flowin’ 🏴‍☠️🦜

When we need to merge things together, what could be easier than using Power Automate.
We have seen that there is a need for visual warning on board ships to be ready if pirates are coming.

Power Automate flow for Hue lights
Triggered 🤔


Therefore, we have connected a Zigbee light chain to a Philips Hue base, on which we use a Power Automate flow to activate a red alarm light.
This is done by calling an endpoint with ALERT in the body.
When the coast is clear again, we can send a “no danger” message and the light will return to green.
It is solved by calling the same endpoint with CLEAR.

Some code for sending alerts to Teams and lights

To make this work, we have connected a Raspberry Pi to the same local network as Philips Hue. We have used a Philips Hue connector from an independent developer.
Fine with everything that is ready-made from 3rd parties 😃

Raspberry PI connecting it all together

Go with the FLOW

The harbor master is a social character needing to speak to as many pirates as possible and in general handle most day-to-day issues at the dock. This means that he is not always in front of his computer and is capable of receiving information on what is happening with the voyages.

Ship ready to leave

When the captain has received his crew from the harbor master and the voyage is ready to leave, the captain closes the voyage and the speakers in the harbor sound out a sweet farewell on the SONOS 🎵Reidar reiser snart🎵.

The flow

Smack my JSON up – and why size matters

Power Automate loves talking to API’s. API’s love talking in JSON. Because why not, we’re in 2023 where curly braces are the equivalent of the human hug – just embrace it already, and grab what you need 3 levels deep. Dot, dot dot. Easy as a dot.

But what happens when you embrace too much? I mean, we’re in 2023 where size has grown (out of proportion) compared to the skinny 70’s. And we’re talking simplistic CSV files here in case your mind was wandering into the land of political correctness. Just saying.

Either way. If the body of JSON is too large, a tad over 0.5MB, Power Automate nullifies it, which prevents you from getting to the property you want. That’s quite the statement.

Fortunately JSON has a string body, and strings can be cut at the front and the back, scissor scissor, slice slice. Let’s strip the away the bloat, grab the good meat, and let JSON be JSON for the time being.

We’re willing to debate if string manipulation is a nasty hack, but this is what PirateGPT has to say about that: “Arrr, string manipulation be a nasty hack in 2023, as it be a shortcut that don’t stand the test of time. It be a quick fix, but be sure to find a more permanent solution before ye be walkin’ the plank!”

And there is no question it’s retro, or as PirateGPT says: “Arrr, ye scurvy dogs! String manipulation may be a tried and true technology, but it’s been outdone by more modern techniques like natural language processing and machine learning. So in 2023, it’s a bit of a retro technology.”