Author Archives: Super INtendo

It’s alive! The final Bananapp™

Posted on by

Soooo. This is it! First off we will try to explain all the functionality and features of our solution. From the working android app to the admin dashboard and statistics for the Road Authority employees and notifications to next-of-kin in case of emergencies. Also we have some comments regarding the four main categories, and assess how we feel we answered them individually.

We had a lot of fun and would also like to take this opportunity to thank the organisers, jury members and all the other teams for an amazing event. Thank you!

That’s-a so nice!

Super mario

What started off as a quirky, fun mario kart related idea, quickly turned into a solution to reduce serious car accidents. Our app detects bananas, but the could, and should, be upscaled to detect other things that can prove a danger. Bananas are indeed dangerous, but there might be even bigger dangers, as we discuss below.

Short technical description:

AI model

We had lots of fun making the training data for the model, from the arts and craft part of it with cut-out bananas, to running around in the snow placing said bananas.

After filming loads of bananas from the car, we fed the videos to our model, and had it identify frames where there possibly could be bananas. We trained it over several days to include different types of weather and degrees of sunlight.

For more details concerning the data and training our model, see hte Data mine(kart) post.

A phone is placed in the car, and when running the app, bananas that appear will be detected. When a banana is detected from the car, the driver gets audio visual clues alerting them to the dangers ahead.

API

When a banana is detected alerts are also sent to our API, running the following steps:

  • User information is fetched from Entra ID
  • Next of kin information is retrieved from a SharePoint list
  • A Power Automate flow is triggered, this handles the following:
    Notifications to next of kin
    Registering incident in Dataverse
  • A map is generated based on the coordinates, and an optimised route is set if there are more than one incident
  • A notification is sent via Teams to the Banana Removal Car Dispatcher, with all data they need (timestamp and location)
  • Alert status is changed sending a request to the endpoint controlling the visual indication in the vehicle

The most impressing thing is that the detection actually works, and we are super proud of our result.

Excellent User Experience
We have two primary user groups. The drivers who run the app on their phone in the car, and employees at the Banana Detection Service. For the first group, less gui is good gui. You run the app, and that is it. We don’t want to distract the drivers with lots of flashy things, just simply give them a feeling of safety, knowing all bananas are detected. For the other user group on the other hand, they need a lot of tools. Both for efficient banana removal, but also to keep their minds busy. We have a simple but beautiful admin panel with maps, games and statistics:

Most Extreme Business Value
The business value we add are actually pretty extreme. Detecting bananas is important and fun, but on a serious note, this detection model can be upscaled to detect actual impediments and objects that should not be hit by a car. Like animals or actual persons. Also – automated reporting to first responders and road safety authorities adds proper, real-life value.

Alternatively, we as a society, can place big banana stickers on everything that shouldn’t be hit by cars, but that seems a bit silly. Maybe just tell people to carry more bananas when crossing the road?

Pandoras Box
Bananas, games and alerts. What can be more fun than that? Also we have more mario mini figures to bribe you with 🙂

Killer AI
This is probably our strongest card, to be honest. We have built our model ourselves, fed it with a lot of self filmed data, and trained it on said data. We are quite happy with the accuracy, on approx. 93 per cent average confidence on banana detection. We have made the model open source so that others can reuse it.

As mentioned above, the model can be trained to detect other things. The possibilities are as many as there are objects in the world. Future uses that seem useful from the top of our heads:

  • Car key finder
  • Wallet finder
  • Jury soft spot finder

Bonus content, footage of us training our model in order to handle different types of weather.

Best regards,

Round 2 of Power User Love

Posted on by

The Banana Portal™  includes a banana pickup route planner for the Norwegian Road Authority. Every detected banana is registered in Dataverse, added as a waypoint, and voila! An optimised route is calculated. Combining power pages (low code) and a PCF written in typescript:

Dreaming in 8-bits

Posted on by

Few things sparks a developers interest more than arguing about languages and frameworks. Which language and framework is new and cool is a very popular discussion in a room full of it nerds. Every day we see new ones appear, and jobs that use late technologies are not popular among young and energetic developers. From Visual Basic through COBOL to Haskell we see that old languages are losing popularity and ultimately dieing.

But exceptions exist… 

Perhaps no programming language has stood the test of time better than C. C is alive in so many ways – theres language derivatives such as Arnold.C, Objective C, C# and C++. C also is what you find under the hood of Python packages like numpy. C also still lives as its own language and is the preferred language for many embedded engineers. But (in our opinion) most importantly, C is the language of the Arduino! 

We use the arduino controller to notify the driver when using our app. We have IoT-ified our solution by connecting the arduino to a wifi-connected endpoint. Because of this, we get to write C. Oh, the glory days. How beautiful to behold this relic that still lives on.

Automating potential uncomfortable situations and (green) shells

Posted on by

Automation galore! A lot of processes are jumpstarted when a banana is detected and the poor first banana responders need all the help they can get. Notifying next of kin can be a tough task, and for Road Authority personell with limited people skills automation is key.

We have set up a workflow that works like this. When a banana is detected we receive a http request. When then register the incident in Dataverse while also looking up next-of-kin data, and send that to outlook to automatic notifications.

Our solutions have a lot of bits and pieces, having control on how these are deployed are essential. All our Azure resources are handled nicely using ARM template deployment.

Whack a banan.. wait is that a duck?

Posted on by

It is the working man who is the happy man. It is the idle man who is the miserable man.

Benjamin franklin

It is important to keep your mind sharp and occupied while waiting for bananas to be detected. With that in mind we have added a whack-a-banana game to keep you on your toes. It uses a canvas, and are embedded in out banana portal. Banana ducks appear randomly in a photo realistic map, and you have to detect it before ti disappears. Come over if you want to play!

(Mario) Party Superstars!

Posted on by

Having started the last lap of the race we want to share some of the fun we’ve had at camp. Both within our team and in collaboration with others. Having been placed in the centre of the room, it has been an utter privilege and an honor blasting mario tunes for everyone’s enjoyment.

We are all rocking matching outfits, even taking into account other people’s nostrils, by bringing more than one outfit!

Other notable features of the camp includes a lot of bananas. Cut-out bananas and 3d-printed bananas, of course! Wouldn’t it be easier to use actual bananas, you might ask. Well of course it would – but then we wouldn’t be able to have arts and crafts sessions!

Camp tour:

We have also given and received help throughout the event. Here are some photos of Herman helping INtendo DS with App Service config, and us receiving help with some prototyping:

General vibe:

Oooh, shiny!

Posted on by

We are fully integrated! In our chop shop we’re creating a motor vehicle testing environment to make sure our applications communicate properly and that the driver is alerted when bananas appear. We don’t want the driver to watch the phone, so we’re removing the alert function from it. Instead, we’re setting up IoT devices to provide the driver with appropriate audiovisual cues. Lo, and behold! The latest patent in the Banana Detection family!

And when you have brought a 3d-printer, you ought to use it, right! We are 3d printing a dashboard that we are integrating a LED light into. The light is attached to a Arduino controller that is governed by a Raspberry Pi. The Raspberry Pi is set up as a server, accepting http requests. When a request is received it alerts the driver that they are approaching bananas through the LED blinking.

Andreas working hard on some 3d-models

Double dash it out!

Posted on by

Literaly, double dash it out. We ❤️ power apps, and want to show you our latest innovation by claiming both Dash it Out and Power User Love.

First off we have created a dashboard with banana-related statistics. We used power pages with the super fun liquid code template language to create a total overview of stats in our admin Banana Portal™ (Hopefully Donkey Kong won’t find the url):

The Banana Portal™ also includes a banana pickup route planner for the Norwegian Road Authority. Every detected banana is registered in Dataverse, added as a waypoint, and voila! An optimised route is calculated. Combining power pages (low code) and typescript:

Edit: added photos of the actual power page.

Star world to the cloud

Posted on by

Choo choo! All aboard the star train to the microsoft cloud! The sole action of finding and identifing bananas is not enough. We need to alert people of the danger that has been detected in traffic before they can cause more harm.

Introducing the Banana Notifier Bot (BnB):

The BnB helps our users by sending teams notifications when bananas have been detected.It uses our notification API (which gets data from our banana-scanning app), Bing Maps API and Graph API with location data and timestamp.

Our Notification API also utilizes the SharePoint REST Api to fetch next of kin information from a SharePoint list. Lastly, we store all our banana findings in Dataverse using the Dataverse Web API. Utilizing a total of four (!) Microsoft APIs.

Data mine(kart)

Posted on by

In order to train our AI-model we need a lot data, but in order to have the correct data, we also need bananas. Being IT people, we never thought of using actual bananas, but instead had a hotel employee in a suit deliver 20 a3 pages of printed bananas that we cut out and glued to pieces of cardboard. Compete with their individual banana stands:

We then placed the bananas in the road around the hotel, got in the car and drove around filming the road from the dashboard inside the car. The conditions were quite foggy and bad, but the enthusiasm was electric. We ended up with a lot of footage including some of a consenting dog and owner.

We then started feeding all the videos to our model, mining for frames that contained actual bananas – our gold.

This is done in a jupyter notebook in python. The first thing the model does is identify all frames that contained potenital bananas, and singling them out. We ended up with over 10 000 potential banana frames. This is one of them with an obvious banana:

The model then strips away everything that is not a banana, in order to get the precise location of the banana:

Furthermore the model does a cutout of the frame around the banana which ends up being used in our actual dataset giving the banana detection app an excellent base and understanding of how to identify bananas.

Actual video from the training data:

Video of banana making:

Photo gallery of Herman and Erik placing bananas and driving around filming the bananas.