Captain’s log – data at your hook-tip

The captain log business value is that he has an app with all the tools he needs at the tip of his hook.

A new feature:

Behold the “Reveal your true face feature”

Revealing the true face

Some pirates find their interest in hiding away, and then it is handy to have an app to show the true face and spot if this is a know fellow shipmate or a rivalling pirate that deserves to be keel hauled.

The service uses a mobile or tablet camera to take a picture of the person in-question and then posts it to mutiny services for identification and face recognition in order to find who this person is.

Claiming the ACDC Craftsman since it is created using actual code with some best practices in react development: https://github.com/TheBastards-ACDC2023/Bastards-NativeApp

Naming the bastards!

Our solution is using Power Automate and HTTP trigger to accept new pirates

And put into a workflow used by our Pirate onboarding app.

Badges

Dataminer – The great database of LLM allows us to mine for awesome pirate names and visual traits for our crew.

Hipster – LLM are so hot, it will be a while until they are not. A multi-billion tech, via Azure Open AI studio cannot be beaten!

Existential Risk – It’s mining, it’s a huge learned model, living on the edge! Sure, we didn’t spend the billions training the model, but that’s hard to do on a pirate’s salary in 2023.

The existential risk explained

Hi Nick, I mean jury 😀

We strongly believe that we have used AI in an innovative way and here are the reasons why. In the fleet enlisting app, we are inserting vehicles into our system with out the need to punch a single character. An AI model has been trained to recognize vehicle type based on picture taken in the app.

Once the app recognizes the picture of the vehicle, the app will communicate with a external database to collect detailed information about the vehicle and insert the data into our fleet management system.

Use case flow
AI Builder custom model
Vehicle tags (objects to detect)
Training images

PirateGPT explained – Simulating mood swings

Travelling the seven seas is great fun and pirates can get as moody as anyone else. The ships current loot holding will continue to diminish in between raids and weather sickness can strike at any time. The below input is what we use to simulate mood swings, also output as logic in code.

Of course, we have no idea if this is the actual code or not, as we ask to get schematized JSON – sanitized for input into the dataverse. With mood data available we can start tracking if a mutiny is far away or close at bay.

We’re thinking the following badges apply:

The Existential Risk – text-davinci-003 has never looked better

ACDC Craftsman – documenting verbatim logic for the logical heads out there just makes sense, ensuring everyone can consume and understand the logic at hand independent of their skillset.

Dataminer – LLM are nothing but a huge heap of external data baby! We’re thinking Babels tower in blob.

External data, external data, external data!

When we’re creating an app for everyone one to use for their everyday Pirate activites coming from F&O, we might as well include som external data as well to make the experience for the Pirate even better. We have never, and never will, argue that you will get all the information you need every day from F&O, so we have given every Pirate a bit more data they can use every day in their canvas app:

  1. When you first log in to the app in the mornig, it’s handy to get a little greeting just “welcomming” you to a new day, right? We have created a PowerAutomate that uses OpenAI to find a new Pirate greeting just for you (if using AI to cheer up your Pirates isn’t an innovative use of AI, then what is?):

2. If you feel down, and just want some text to make you laugh a little, you don’t need to go to the internet, we have created an Insult of tha day feature in your app. Just click it and start feeling better, at least if you think that this insult is regarding you bunk-buddy 🙂

3. And then, if you want to know what kind of weather it is, look no further, it’s already right there in you Pirate app:

The Existential Risk – using AI to detect potential recruits and giving them proper pirate names

Pirates 365 uses several AI components

TensionFlow

We are using Tensorflow to detect passing pirates

Tensorflow can run in the web browser, and is able to both detect persons and other objects like bottles.

Hooking up the webcamera with the frame analyzer
Detecting bottles and pirates

ChatGPT to create PirateNames

We have already blogged about how we are usting OpenAI’s DaVinci model to generate Pirate Names here

Here we use the following prompt

Create a creative and cool pirate name from the name "{firstname lastname}".  Do not use the word "captain". Answer only with the name - max 3 words. Only output text characters a-z, no special characters!

Hipster-ish technology

Making this we leveraged vue.js, pinia, three, troisjs, tensorflow, ChatGPT. The list goes on 😉. All pretty hip, (almost) non-ms technology.

Badges

We hope this sholud be sufficient for getting the AI- and Hipster badge 🏴‍☠️✌️🦜

Capture new pirates

When we need new pirates on our crew we have an app where we can take their picture, and the app send the picture to a server that uses AI to piratefy your face.

The app is a Canvas App with a camera control on the first screen

When user click the camera control they navigate to the next screen where they will see their picture.

They can go back and take a new picture if they are not happy, or move on to “Piratefy me”.

If user click the “Pratefy me” button a power automate flow is triggered. We pass the image base64 value as a value and store the reply value in a variable.

The flow is “Me as a Pirate”

The trigger is a power apps trigger.

First we initiate reusable variables

We grab the image value from the Power Automate

Passing the image over to the CLIP service powered by Stable Diffusion at the Frogner/Oslo datacenter we annotate images to mostly correct the gender or non-gender of the person in the photo. This is our responsible way to ensure the pirate representation of the person is not turning the most beautiful guy and girl into something they definitely don’t identify with.

We parse the response we get

Next we pass in the captured photo, and overlay it with some serious pirate’y prompting to ensure our new crew mates meats the bar (and also the real bar when port-side).

We need to modify the image string to make it the right format for the Ai to handle.

We trigger a child flow (documented further down)

We gather the reponse and send it back to the Power App.

The flow triggered as a Child flow is called “get Crew Member Data – PirateGDP”

The core API in out solution is PirateGPT, a large language model fully capable assigning awesome pirate names and corresponding ships, with the identifiable traits of the pirate. Which we as a responsible HR ship keep fully secure and up to date as needed. When you have seven John’s on one ship, it’s quite important to distinguish them by traits such as red beard or cock farmer.

We PARSE the results we get for get the right format

Respond with the values to the “parent” flow

The Power App

Back in the Canvas App the user will see the newly created name and their picture Piratified:

It also navigate to the next screen where they will see the pirate name they are given and their picture Piratified.

If this isnt glossy and extreme business value, I don’t know what is?

Imagine this being a manager taking pictures of new employees. We all know that Gen Z are used to looking at themselves with prettifying filters. This way the old middle manager can take pictures of their new employees and let Open AI add the familiar filters so that when the Gen Z employee see their own new profile picture they will actually recognize themselves.

Badges

Glossy Pixels – because pirates are beautiful.

Go with the Flow – because it’s all flowing left and right

The Existential Risk – Who knew LLM could be used to generate schematized JSON data structures, and who knew all those crazy internet pics would be the basis for pirate faces of 2023?

ACDC Craftman – All API’s are tested and documented in a shared Postman collection, all is ran over https (not from the beginning) to avoid photo snooping, and API keys are used as the extra layer. No one wants to be the next TMZ victim!

The Pirate Assistant

This is every pirate’s best friend and a must have!! Loaded with awesome features this killer app should be installed on the mobile phone of all pirates in your organization.

Targets

Get a quick overview of your targets. Once a ship has been become a target in Dynamics 365, you will be able to see it in your mission overview. This is perfect when you have a long day of plundering i front of you and need to know which ones to do next.

Rumglasses

Have you ever been too deep in the rum bottle and close to do something stupid with what you believed to be a mermaid out on the seven seas? Or maybe you actually did and are now in for a hard time facing your mates with humiliation and regret?

Not anymore! Rumglasses let you take a picture of your believe-to-be mermaid and tells you if you are right or not. By training an AI to recognize and distinguish manatees from mermaids you can now relax.

My Ship

A map showing your location when out on the seven seas. You will se both your ship on the map and all other ships in the area. This is how you know you are close to your target when out on a plunder. The best part is that military or law enforcement ships will occur on the map as well so you can wait until the right moment before attacking the target.

Bottle Mail

Communicate with your mates by using bottle mail – a feature for sending email when you are on the fly. It suggests recipients in you address book or you can write to a pirate outside your organization. This way, you don’t have to switch to another app (email client) to send messages to your crew.

Pollybot

You are not a real pirate unless you have a parrot. Unfortunately (or luckily) there are laws about how to treat your animals today that did not exist in the old glory days. It might not be appropriate to bring it with you on a raid :/ This is where Pollybot comes to the rescue! Your own digital parrot who you can talk to, ask questions and keep you with company. As a real parrot, it sometimes answer your questions, but most of the time you will get an insult or just nonsense in return. But that doesn’t matter! Where a real parrot will fly away to save itself, Pollybot will always stay close.

Mayday, Mayday

When Jack Sparrow was stranded on a desert island, he managed to escape by tying his back hair together and catch a couple of sea turtles who gave him a ride back home. As fun and comfortably that sounds, not all pirates are equipped with back hair that long. However, most of us got their phone in their pockets at all times!

Whether you are stranded at a desert island, your ship is about to sink or you are stuck in a safe full of treasure, this feature will save you.

When using the mayday function your coordinates are then sent to a email address monitored by you crew so they can come and pick you up. If your mates prioritize your request is however another matter.

Because of the fancy design, the use of AI and the huge amount of useful features in this app, we hereby claim the Glossy pixels, Existential Risk and Feature Bombing badges. We also claim the Power user Love because the app is created in Power Platform 😀 #ProCodeNoCodeUnite

The Pirates 365 Recruit – Pirate recruitment terminal (PRT)

Recruitment is an important, and integrated part of Pirates 365, to acquire the best talent for our crew. To help us with this we have developed The pirate recruitment terminal (PRT). This solution provides us with a visually appealing and an engaging experience to gather new recruits.

Illustration of the Pirate recruitment terminal’s place within the Pirate 365 Recruitment module, along with the other modules.

PRT User flow

Since pirate recruits normally aren’t that familiar with modern interfaces, we have chosen a living parrot to help and guide the user and provide a smooth user experience.

When a pirate is detected, our engaging parrot approaches the pirate and presents the sign-up option. We collect personal details and complete the process by taking a picture. The collected data is then sent into the platform for future processing.

Sign-up form
Picture capture

The Pirate recruitment terminal technology stack

PRT The app is running on the vue.js progressive javaScript framework with Pinia🍍 for state management. For the 3d animated parrot🦜 we are using three.js and TroisJS with WebGL rendering. For AI vision👁️ and object detection🤠 we are running tensorflow under the hood. If available the model is running on GPU, if not we fall back to CPU.

Images, 3d model, fonts is stolen captured on the internetsea.

Everything is running in the local browser🤓

Mates Upcoming Raid…

Eyyrr mates!!! 

Captain Black Bart, is planning a big raid, Oslo is the target and according to ChatGPT the loot potential is #awsome.

Invitation is sent to all chosen crewmates using top of the art “Power Automate flows”. 
Crewmates are free to Accept or Decline the raid invitation.   

When it’s time for the raid, we requires all crew members who have accepted the raid to check in, using their RFID card or phone. 

If the crewmate have accepted more than one raid, he/she can choose to check in using our Raid Planning App 

With the help of Midjourney AI, https://docs.midjourney.com/,  we generated the app design component by asking the AI to provide us witht the latest & greates piracy themes.  
 

The RFID data for all potential crewmates is stored in Azure AD using the “Postal Code” field, pretty “Nasty Hack” according to our “bosun”. 

This data is imported to Dataverse “AAD User” virtual table for our app. 

Let’s raid mates.