Not reinventing the (kart)wheel

Time is of the essence when there is undetected bananas about. We will probably spend some time training our algorithm to detect all sorts of potential road impediments, but in order to rush to market and deliver value as early as possible, we instead grab a mushroom and cut some corners. Much like Boo in Mario Party, we can steal other people’s hard work!

We started by forking the Tensorflow repo to make the basis of our app. Tensorflow is an open-source end-to-end platform for machine learning, and a great starting point for our app. We can use it to create models for machine learning. And while speaking of thieving, Tensorflow is also built on other public packages – also a form of stealing.

Speaking of models, why make your own when there are several detection models we can “borrow”

Mwa ha ha! Surprise, Luigi! It’s-a me, King Boo!

King boo – Luigi’s mansion 3

Others have also created several models for mobile use that we can reuse while we try to train our own. MobileNet model is one of these and can be used straight out of the box. These models are in turn also trained on public data that we can “steal” and modify as we please.

We also “borrow” more tangible things. Like non-licensed photos of bananas for training our detection model and map and geo-services for location data.

TL;DR Thieving is good – at least in programming

Ask and you shall recieve

1. Customer Service Portal
To create the ticket/case that should be resolved. Add live chat support using real time Communications. We will be using Socket.IO as technology.
2. The Kiosk App and Device(iPad) is powered by AI and based on the canvas app.
Business needs: Every solution requires some Marios(agents) to help the princess solve the problems.
So, our kiosk allows you to create Mario’s profile:
The avatar, skillset, and story will be generated based on your photo, and then Mario’s profile can participate in solving the cases for the princeses. (no personal data will be collected)

3. Model-driven app:
Provide the opportunity to match the agents and manage the case-solving process in a more advanced mode.

4. Clippy

We traveled to past and get back with Clippy. Using PCF controls we managed to implement Clippy in model driven app. Our original plan was to connect Clippy to AI and use it to help users. When the App is updated and the user logs in for the first time it will show a notification and highlight changed parts.
We were so impressed by Scott’s video of the copilot that we decided to add some familiar and modern features to the model-driven app. So we are glad to announce Clippy – your GPT-powered copilot. He will help you establish the most optimal process by providing insights based on AI. We found a very nice looking github library for implementing clippy with pcf.

Thieving the solutions with Orange Bandits

We decided to ingest information from Norwegian National Registry of Vehicles (Statens vegvesen) in order to limit the information Pirates need to input when onboarding the vehicle in our app:

We can use this information to get the Make, model, registration data, technical check, etc… Basically, anything that is needed to be known to thieve the vehicle! And this is just by query by registration plate.

Arrr, the scrawp (script) be mighty powerful in this one!

Pirates love talking trash, and where better to spew out all the fun than Microsoft Teams?

As a pirate you don’t have too many tools at hand at any given time. The same is true in 2023. Why learn something new when you have your trusty old swiss army knife at hand?

This time around we’re stealing from our own team mate, Mikael, taking one of the most powerful and mis-used plugins of all time, the PnP Script Editor Web Part, and we use it to easily embed our “Pirate “Capture a pirate” canvas app into Teams.

Teams App Admin

It’s as easy as rum and grog, add the Teams Tab, configure it with and <iframe> tag and you’re all set. Because everyone knows iframes as it has been a trusted valid tag for over 20 years now. Who knew you could embed and use a PowerApp within Teams????

Script editor web part – iframe

Once set up – it works beautifully!

Canvas Apps in Teams – such a novel concept

The beauty is that it even works on mobile! Pick the tab and embrace.

Capture a pirate responsibly on a device

And for some extra fun, here’s PirateGTP pirate in SVG – well hidden 😁

The input
The output

Badges and why – because why not

Client side salsa – Everyone knows teams tabs are browser tabs, and iframe’s are all about browser love. No monolithic front-end used, only simple and easy html and scripting.

Thieving bastards – We stole and borrowed the open source PnP script editor web part solution, even though inception was involved. PirateGPT is our API on crew mate generation, and Stable diffusion (via automatic1111) is our image pirateer solution. And not free I tell you as we had to purchase a used Nvidia RTX3060 gfx card to get this going.

Chameleon – What can we say, Teams, stand-alone, browser, native app, what not – you will not get away from being onboarded with out solution. It adapts portrait, landscape, even black and white. PirateGPT will get you!

Plug’nPlay – We stole it with inception, it plugs into anything SPFx plugs into. Showcased in Teams as we just love the purple.

Power User Love – All stored in Dynamic, all surfaced in Teams and SharePoint – we love ya’ll!

The Pirates 365 Recruit – Pirate recruitment terminal (PRT)

Recruitment is an important, and integrated part of Pirates 365, to acquire the best talent for our crew. To help us with this we have developed The pirate recruitment terminal (PRT). This solution provides us with a visually appealing and an engaging experience to gather new recruits.

Illustration of the Pirate recruitment terminal’s place within the Pirate 365 Recruitment module, along with the other modules.

PRT User flow

Since pirate recruits normally aren’t that familiar with modern interfaces, we have chosen a living parrot to help and guide the user and provide a smooth user experience.

When a pirate is detected, our engaging parrot approaches the pirate and presents the sign-up option. We collect personal details and complete the process by taking a picture. The collected data is then sent into the platform for future processing.

Sign-up form
Picture capture

The Pirate recruitment terminal technology stack

PRT The app is running on the vue.js progressive javaScript framework with Pinia🍍 for state management. For the 3d animated parrot🦜 we are using three.js and TroisJS with WebGL rendering. For AI vision👁️ and object detection🤠 we are running tensorflow under the hood. If available the model is running on GPU, if not we fall back to CPU.

Images, 3d model, fonts is stolen captured on the internetsea.

Everything is running in the local browser🤓

Mates Upcoming Raid…

Eyyrr mates!!! 

Captain Black Bart, is planning a big raid, Oslo is the target and according to ChatGPT the loot potential is #awsome.

Invitation is sent to all chosen crewmates using top of the art “Power Automate flows”. 
Crewmates are free to Accept or Decline the raid invitation.   

When it’s time for the raid, we requires all crew members who have accepted the raid to check in, using their RFID card or phone. 

If the crewmate have accepted more than one raid, he/she can choose to check in using our Raid Planning App 

With the help of Midjourney AI, https://docs.midjourney.com/,  we generated the app design component by asking the AI to provide us witht the latest & greates piracy themes.  
 

The RFID data for all potential crewmates is stored in Azure AD using the “Postal Code” field, pretty “Nasty Hack” according to our “bosun”. 

This data is imported to Dataverse “AAD User” virtual table for our app. 

Let’s raid mates. 

They see me flowin’ 🏴‍☠️🦜

When we need to merge things together, what could be easier than using Power Automate.
We have seen that there is a need for visual warning on board ships to be ready if pirates are coming.

Power Automate flow for Hue lights
Triggered 🤔


Therefore, we have connected a Zigbee light chain to a Philips Hue base, on which we use a Power Automate flow to activate a red alarm light.
This is done by calling an endpoint with ALERT in the body.
When the coast is clear again, we can send a “no danger” message and the light will return to green.
It is solved by calling the same endpoint with CLEAR.

Some code for sending alerts to Teams and lights

To make this work, we have connected a Raspberry Pi to the same local network as Philips Hue. We have used a Philips Hue connector from an independent developer.
Fine with everything that is ready-made from 3rd parties 😃

Raspberry PI connecting it all together

The Skeleton Crews Skeleton Sketches: Hack Sparrows Architectural Treasure Map

When first presenting our idea, the Hack Sparrows shrouded it in mystery. A battle tactic as old as the kraken herself. In the words of Sun Tzu: Never let you enemy know your plan, at least if you have one. Which we most certainly have…

Initial sketch of what The Hack Sparrows wants to achieve.

The Traveling Pirate

Ever heard of the traveling salesman problem? It is a mathematical problem of route optimization: A salesman travels form A to B. In between he has many opportunities for trade. How can we optimize the route in terms of monetary value for the salesman: Which patch gives most bang for the buck, or most buck for the step?

The Bare Bones

We are no salesmen, we are pirates! And based on the presentation many of us are after the same treasures, leaving the best planned pirating rout the winner of pirates. The pirate king!

Our cleverly designed and thoroughly QA’d Architecture (do not double check) shows how we will use state of the (last century) art to enable better piracy.

System diagram showing what the user sees and what the system does. Proudly made with a thieved licenses from our employer. Yarr! (we have license, we just act like we stole them)

On the seven seas the captain has access to a web interface. This web interface uses a Function App to talk to two main services:

  • We data mine the Barenswatch.io service, which keeps track of all commercial vessels on the seven seas, including the Privateers. And store them in our Data Verse,
  • The Data Verse updates and stores the crawled data to keep track off ship movement, ongoing raids and opportunities (including flagging Privateers as hostile)

Outside the web interface we use tactile feedback systems powered by Arduino (or Our Raspberry Pi is we can get it working) to warn the user of dangers or opportunities.

Henrik creating the Data Verse tables for the power platform that will give power to platform, empowering the pirats to pirate commercial platform vessels

    Full solution from PowerShredders of Axdata

    We came into this Hackaton with the idea that we could make the Onboarding process a lot less manual, a lot more secure and reduce the risk of human errors in the process significantly. And we’ll be honest, we are extreamly happy with the solution we now are traveling home with! The best thing you ask? It works like a sharm, every bit of it! We have tested it several times, and everything flows exactly as described!
    You can see the full solution under here, but we’ll start with a summary of why we think we should get a lot of points in the four different main categories:

    Excellent User Experience:

    Lets start with the help this will give to the employees already working in the company:
    – HR now will have a veary streamlined process for Onboarding, where all tasks that can (and should be) automated are just that. They don’ need to spend time sending out agreements, follow up with signature, enter a lot of information regarding the new employee in the system, follow up on other employees that forgets the tasks they have in an Onboarding. They don’t need to notify IT about a new employee coming it and wait for creation of user and access to systems. All of this happens automatically. Nothing is forgotten, no sensitive information is sent in emails or seen by someone who shouldn’t see it.
    – IT never needs to think about a new employee coming in anymore. Everything is automated and just happens. Isn’t that the best user experience? When you actually don’t have to do even one click, and the process still works?
    – Other employees in the company having tasks regarding a new employee coming in will be reminded of their tasks and make sure nothing is forgotten. Automated and nice. And, if they complete their task, no notifcation will be sent, it really is as easy as it sounds

    And then, to the candidate starting. If the company implements this solution, everything will be ready for the new employee when he has his firsgt day of work. He can even get information about the company, his team, his manager and more in the Onboarding Portal before he starts, so we can keep up the good energy people feel when they are about to start working at a new company. The new employee will also feel that this company really takes care of their employees, and that they really are up to date in the digital world we’re living in.

    Most Extreme Business Value:

    The value for companies here are so high, that it’s almost difficult to know where to start. But, first of all, this saves a lot of time for the HR department. And, we really mean A LOT of time. Not just that everything is automated so they don’t need to do as much as they have to now to register a new employee, but they don’t have to push and follow up everyone else that hasn’t done their part and they don’t have to correct human errors that’s been done during the process. They can spend their time on something that is much more valuable to the company, and that is to make sure that all the employees already working here gets the best environment possible to do their job as best as they can! Lets face it, this is what we want the HR department to do, we don’t want them to spend time on entering data into a system.
    The IT department will also save a lot of time witht this solution, that they can spend on other things as well. Not that I work in our IT department, but I assume that creating users and assigning licenses and access aren’t the most fun task they do at work. So I would think this will actually make their workday more fun!
    Let’s just do an approxemently calculation of time saved. Create a contract for the new employee, upload it for e-signature, send it for signing, recieve it, ask the employee for more information so that it can manually be put into the system. Let’s say this totally takes 5 hours at best. Then someone needs to tell IT to create user and access and make sure that it have been done, a total of 1 hours. Then someone needs to follow up other employees, to make sure everything is done and ready for the new employee. This is maybe the most time consuxming, and I don’t think it’s wrong to say that this takes at best 8 hours in total during the time between someone signs and their first day of work. This means that time saved for just this one onboarding process is at least 14 hours. One of our customers typically hires between 10 and 20 employees every month, and this means that this solution will save them between 140 and 280 hours. Each month! To have an other example, an other customer of ours plans to hire 1.500 employees by the end of 2023… You do the math here!

    Rock Solid Geeknes:

    There are so many partws in this solution that works perfectly together, we almost can’t believe that we actually made this work as good as it does now. We seriously din’t think that we would be able to automate that many parts of the process as we have done now.

    Killer App:

    We have created this as a total solution, so that everything can be uploaded and used by all companies using Dynamics FO og Dynamics HR. With only a little personalization to make the wording, logo etc. fit the different companies, this can be set up quite fast. And, of course, each part can be implemented by it’s self too, if someone don’t want the whole solution. But, we know that this will be an easy solution to sell to our customers, as we know that a lot of them already really wants this!

    The solution:

    And now, finally, let’s go to the solution. You have been through a recruitment process in your company, and you have picked a really awesome candidate you want to send an offer to. This is what you do:

    1: Create the candidate in Dynamics, and fill name, startdate, email address and phone number, and connect him to the position:

    Like magic, the candidate resieves an email to view, update and sign the agreement. The Agreement is automatically filled with data from Dynamics before it’s sent out for signing, done through an API we created with OneFlow right here at this Hackaton:

    The candidagte fills in the National ID number, and signs the Agreement using Bank-ID:

    3: Agreement is signed by the company as well:

    4: You Hire the candidate in Dynamics with two clicks, and assign the onboarding checklist that should be used for this canidate, and ticks the checkbox to create a user in Azure AD for the new employee:

    This triggers a PowerAutomate that first create a work email for the new emplyee, sets this new email as the primary email address on the Employee workspace in Dynamics and creates the Azure AD user for the new employee and assigns group and licenses to it, and ends with sending the login detail to the employee with SMS, together with the link to the Onboardig Portal:

    And, of course we have created a PowerAutomate that will remind the employees in your company about the tasks they are assigned regarding the onboarding of the new employee if they don’t follow it up by themself:

    Now, the employee can log in to the Onboarding Portal (that works on all devices), and enter in more information about himself. When submitted, the data is automatically updated directly in Dataverse, and then becomes visiable in Dynamics within seconds:

    And, last, but absolutely not least, the night before the new employee has his firs day of work, a PowerAutomate runs and imports the user to Dynamics, connects the user to the correct employee in Dynamics, and assigns the security role Employee to the user so that the new employee has accsess to the Employee Self-Service workspace in Dynamics:

    And, to end it all for now, we have automated the process for email signature, so that is is automatically put on every one of the emails you send out. Now they finally will lokk the same for the whole company:

    The signature itself is created with HTML-code, after a rule is created in Exchange