Adding realtime streaming to Pirate recruitment terminal (PRT)

Our PRT is doing real time object detection to spot pirates and bottles, but the data doesn’t leave the terminal. As the captain wants to know how many recruits we are attracting we needed to expose it to his “weapon of choice”; Power BI.

Luckily as the app is using a modern data store🍍 this is easily achievable by inserting a listener and sending the data to power bi on change 👍.

The way we connected to Power BI was to go to “My workspace” and create a new dashboard which has a streaming dataset which connects to an API. Then we gave inserted the right values from stream in order to get the right data.

We are now getting the real-time data into Power BI, we just need to add visuals into the dashboard. We go to “Edit”, and add a tile by using the real-time dataset that we created. We added two gauges and two line charts.

These visuals will now give real-time metrics from the parrot when there are either faces or bottles detected from the camera.

Right now collaboration

Canvas apps “recently” introduced the ability to work 2 people on a single canvas app. This has not been available before and is absolutely a gamechanger for app development!

We started out by turning on the function in preview from settings

After this, we have to connect the app to a GitHub source. I chose to connect this to my personal GitHub for simplicity.

At this point we were not able to work together on canvas apps without having to close the app between changes!!

This is a big change for Canvas apps being able to work at the same time in the same app👏

BombShells – right now or later

I Oslofjorden er det plassert ut sensorer for å detekere forurensing.

Sensorene er koblet til en pc og kommuniserer mot port COM3. Vi har derfor utviklet en tjeneste som kjører lokalt på pcen og lytter på COM3. Når sensorene detekterer en hendelse så sendes det en melding over port COM3 og servicen våkner, validerer input og sender informasjonen til Azure Event Hub.

Årsaken til at vi har valgt å benytte Azure Eventhub
1) Vi ser for oss at vi på sikt skal motta mye sensor data i realtime
2) Kommuniksajonen skal kun gå en vei, dvs fra sensorer til eventhub

Det utføres en validering om Turtels skal sendes ut og redde miljøet.

Når turtels er ute på oppdrag ønsker vi at de skal kunne kommunisere med hverandre. Derfor hatr vi tatt utgangspunkt i eksempel app og modifisert denne til å funger fort vårt case.

azure-signalr/v1.md at dev · Azure/azure-signalr · GitHub

og satt opp infrastrukt i azure

Med dette ønsker vi badges right Now

Together we get the best solutions!

We started this mornig with a call to a company called OneFlow. We have had some conversations with them before, and decided that we wanted to try to get something done together with them today. We wanted to bulid an API integration from Dynamics 365 HR to OneFlow, so that we can sen out Employment Agreements for digital signatures directly from Dynamics! If this is not a huge business value, then we don’t know what are!
This is what this we did together with them right here and now, and how it works:

Contract is generated in oneflow, and is sent to the company representative as well as the employee.

When the contract is signed electronically by both parties, power automate get a callback with information on which employee that has signed and we can tick off “contract signed” in HR and add an url link to the contract in oneflow.

The solution is set up to support Swedish BankID,

Danish NemID, Finnish BankID,

Norwegian BankID and advanced e-signature for other countries with e-mail/phone verification

Setting up the projects backbone

Azure Cosmos DB

The solution is using CosmosDB to store Users and Telemetry coming from the different devices/assets

CosmosDB is fast and easy to maintain for noncomplex solutions. The main use here is for very simple data structure and for mass data allocation. Since we are getting data from this DB in real time, so we need to send a lot of requests in very short time ranges. Since CosmodDB is DocumentDB based so it handles these kinds of situations smoothly.

Azure WebApplication

We have created a web application as the user interface to keep track on the assets. The web applications is based on .Net Core 6 with React/TypeScript. We have implemented some of the main .net core features as:

– Entity Framework: To connect in a simple way to CosmosDB

– Controllers: As API endpoints

– Azure KeyVault Provider: As a provider, the same as an appsettings file. This protects all of the secret variables like codes, keys, connection strings and so on.

– Session: To create an encrypted cookie (Https, SameSite) to keep the session alive

– Cors: to prevent other origins to send requests to the BackEnd

– Dependency Injection: To inject all different services to the solution, including custom services and custom repositories.

– Hub: To connect to the SignarlR applcation

Azure Maps

Azure maps is used to track the geo location for all different assets. It will show the position of all assets in real time with the help of Signal R.

We have given a little bit of extra styling to the pointers and added some of the out of the box controls.

Azure Signal R

We have implemented Signal R to open a websocket for real time communication. The main idea was to avoid so many common http requests to the backend which is not so good when it comes to performance and security.

The front end sends requests to the Hub and it immediately gets a response so the data is always fresh and new. This is the way we keep track on the assets by getting all the telemetry/data needed from the Cosmos DB.

PT Brixters – Conductor Portal (Badge claims follows)

PT Brixters have made a conductor portal in react.js that is fully responsive and user friendly !

The Portal is made in React therefore we are claiming the “Client Side Salsa” badge for this.

The Portal also uses 2 module apps to show the train live video feed and a train controller where the train controller uses socket.io to communicate from the frontend to the backend, therefore we are claiming the “Right Now Badge” for this.

We are also claiming the “Glossy Pixels Badge

for the conductor portal because the portal looks shiny, is easy to read (big text, big buttons). And its responsive, works on mobile devices also, therefore we are also claiming the “Chameleon Badge”

And we wanted to show of our old Dashboard that is now upgraded to the new one 😀

Some pictures of socket.io with flask in action for the “Right now badge”:

Picture of VS Code as proof that we are using React to build our app.

“Near-Time”-data fra bonden

Et gammelt indiansk ordspråk lyder som følger (ikke verfisert): “Før du rekker å få sende en lang beskjed via røyksignaler, kan det allerede være for sent”. Dagens bønder har behov for oppdaterte data fortløpende og i vår løsning for vekstkontroll har vi tatt i bruk teknologi som understøtter dette behovet.

Sentralt i løsningen finner vi en Event Hub som kan ta imot data fra IOT sensorer eller fra bildeanalyse. Event hub’en kan videre supportere mange konsumenter av data som følger strømmen i sin egen hastighet.

Som i vår løsning skissert over så inkluderer dette Stream Analytics for analyse og aggregeringsformål, arkivering til andre datalagringstjenster slik som Azure Table Storage eller en .NET Core app som direkte konsumerer og behandler strømmen av data fortløpende.

To enkle Stream Analytics Queries som går leser fra Event Hub og sender data inn i et PowerBI dataset (cowders-sensor2) og til Table Storage (cowderstreamoutput) for historikkformål

Claiming badges

“Right Now” for designet og implementasjonen av en Near-time løsning

IOT: Reading out messages on Sonos from Dataverse

When tasks are created in our nice little family the kids often doesn’t respond when their parents are sending out push notifications to their devices.

As a way to remind the kids on their tasks we have developed a system that reads out the tasks on the sonos speakers in the house

This setup requires the following components

  • Dataverse Tasks table
  • Power Automate Flow with “Common Data Service (current environment)” and “Service Bus” steps
  • Azure Service Bus
  • Talkity.com free text to speech subscription
  • Some device able to run Python on the same local netwok as the Sonos devices. E.g. a Raspberry PI
  • One or more Sonos Speakers

Power Automate Flow to put messages on Azure Service Bus Queue

Text to speech using Talkify

We have investigating several services for Text to Speech. Azure Cognitive Services have some, but we went the easy route and found one where it was possible to just compose a long url and post it directly and get a MP3 file with the speech back. https://talkify.com

The URL is on the format https://talkify.net/api/speech/v1?text=Clean your room&rate=2&format=mp3&voice=Microsoft Hazel Desktop&fallbackLanguage=English&key=xxxxxxx

The paid verison of Talkify also supports Norwegian, but as we are using the free version only English are supported

Our initial idea was to have a power automate flow step to do fetch the MP3 and upload the MP3 somewhere readable for Sonos (like Azure Blob Storage), but when it was as easy as calling an GET URL we can send that URL directly to Sonos.

Subscribing to the Azure Service Bus Queue and triggering the Sonos Speakers using Python running on a Raspberry PI

So playing files on the Sonos isn’t THAT difficult – especially when the sound files are from publically available URLs

The following Python script is using the SoCo Sonos Library (https://soco.readthedocs.io/en/v0.21/releases/0.13.html) and Azure Sevice Bus SDK v7.

The python script is deployed to a local raspberry Pi that can works as a local controller of the Sonos System.

Note: as long as you are on the same network as a Sonos speaker you can control it without any authentication. Tip for practical jokes 👌👍


from azure.servicebus import ServiceBusClient, ServiceBusMessage

from soco import SoCo
import soco
import urllib

sonos = SoCo('192.168.x.x') #kontor
print(sonos.player_name)

with ServiceBusClient.from_connection_string("Endpoint=sb://acdctaskservice.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxx=") as servicebus_client:
        print("ok")

        while True:
                try:
                        receiver = servicebus_client.get_queue_receiver(queue_name="taskqueue", max_wait_time=5)
                        with receiver:
                                for msg in receiver:
                                        print(msg)
                                        sonos.volume = 10
                                        url = "https://talkify.net/api/speech/v1?text="+ str(msg) +"&rate=2&format=mp3&voice=Microsoft Hazel Desktop&fallbackLanguage=English&key=xxx"
                                        print(url)
                                        sonos.play_uri(url)

                                        track = sonos.get_current_track_info()
                                        print(track)
                                        receiver.complete_message(msg)

                except Exception as e:
                        print( "Error: %s" % e )	
Bilderesultater for raspberry pi
Bilderesultater for sonos 1

Hopes for (some of) the following badges:

Embedding Numbnut

Go with the flow

Right now (uses service bus to send events directly)

Thieving Bastards (uses 3rd party SoCo Sonos library and shady text to speech service)

Nasty hacker (sends in the composed text-to-speech url with subscription key and everything to sonos)

And maybe a point or two in the categories

Blow my Mindstorm

Lego Autobots

Lego Shark Thank