Tag Archives: Dataminer

Crashing the Azure AI Studio and Copilot

Posted on by

Badges to claim:

  • Nasty Hacker – for retrieving data from 3rd party services and syncing it to the blob storage to connect to Azure AI Studio
  • Data miner – for retrieving the case information from dataverse and calculating the average score using AI.
  • Embedding numbnut – for embedded copilot in Model-driven app
  • Stairway to heaven – for using Azure AI Studio, Copilot, Blob storage, and in previous articles also Azure Function, WebApp

Our solution includes the latest features of the PowerPlatform and Azure connection the Low and Pro code approaches together, to allows you to boost the performance of resolving the cases by using some insights from the Copilot. 

  • The business use case is about complicated cases when we need external consultancy and assistant, so task is to find the suitable Marios-consultants according to the customer request, by searching the professionals on the Indeed, comparing their background and experience and finding the best matches. (to find the suitable Marios according to the request.) 
  • The business use case is about simplify the KYC(Know Your Customer) process by using unified workspace for all operations.  From Indeed we can understand the company background,do semantic analysis of the comments to have insights on how technicians can approach customer (princess) (simplify the KYC(Know Your Customer) process by using unified workspace for all operations.) 
  • The business use case is about analyzing current princess (customer) and her needs based on the Indeed. The data that we were able to retrieve contains open job postings, company in for, recent news and personal profile info. Based on this information, we can suggest more other services and provide information to sales and marketing departments. (analyze current princess and her needs based on the Indeed profile and suggest more other services.) 

We deployed GPT-4 model to our personal instance using AI Studio, then fine-tuned it with company’s internal data and data from open sources like Indeed, Glassdor or proff.no. 

Than using Retrieval Augmented Generation (RAG) technic to inject generative responses from big LLM into answer of Copilot. 

With RAG, the external data used to augment your prompts can come from multiple data sources, such as a document repositories, databases, or APIs. The first step is to convert your documents and any user queries into a compatible format to perform relevancy search. To make the formats compatible, a document collection, or knowledge library, and user-submitted queries are converted to numerical representations using embedding language models. Embedding is the process by which text is given numerical representation in a vector space.  

AI Studio has seamless integrated feature use Azure Blob Indexer to implement search functionality search for AI. It bring the possibility to simplify the access to the Datalake from LLM side.   

By implementing multiple connectors to the third-party services and data sources, together with Dynamic chaining feature of the Copilot it gives cleaner user experience for the user using only one tool for analyses. 

How it works:

Mining into Peach’s Kingdom

Posted on by

With this post we claim the badges:

  • Dataminer – for using dataverse as a datasource in Power BI, with data from the games.
  • Plug N’ Play – for embedding the Power BI report into teams.

The data from the Power BI report is coming from a table in dataverse called “Leaderboards”. You have the score of each player for a certain game, and what number of try there was for that game. We then use this data in the Power BI report to show all of the players score in a dashboard. There is also the possibility of seeing one specific players data. Which is of course presented in the same graphical theme as the other apps.

And of course it doesn’t stop there. We then also made sure to embed the Power BI report into Teams. Into our Peaches-team where we can then keep track of the players progress. The report addresses a crucial business need by allowing us to efficiently track and analyze the progress of players in the various games. We can then make data-driven decisions, like seeing which game the kids find more fun and entertaining. This can then be used to further develop games for the kids, and enhance their learning experience. Which will allow us to constantly evolve and improve.

Piping our way through the Azure Cloud

Posted on by

To make our amazing service Tubi work, a lot of cloud is needed. We aim to make the plumber’s job easier by recommending the best layout for where the pipes should go, and for that, we need AI. We have trained a model in Custom Vision to recognize all components in a bathroom that need water. So, when the plumber uploads a floor plan to our Static Web App, the image is sent to our Azure Function App backend in C# Asp.net through our own API. But both the image and the equipment list must be stored somewhere. Therefore, we have also connected to Azure Blob Storage. Then last but not at all least. The people working in the back office have instant interactive reports available to help them with filing and billing through Power BI and alerting the using an automated flow (Badges: Feature Bombing)

Sometimes it works, and that’s plenty

Databases are good, but sometimes it’s easier to just dump everything in one place until you need it again. Yes, it might not be very scalable or very normalized. SQL became too heavy, and we already needed a Blob storage to store the images, so we also dump the order data in the same blob storage as JSON files. It’s old fashioned way of serverstorage, and a bit dirty, but it works! (Badges: Nasty hacker, Retro badge)

Power the backoffice

As the final list of components are decided, they still have to be approved from the accounting team in the office. To make sure they have all the information they require, we have developed a Power BI dashboard to crawl through our registered data and make sure the orders are handled properly (Badges: Crawler, Dash it Out, Dataminer). And to make sure the orders are handled easy and fast, the dashboard is embedded into teams and an alert is automated by using a logic app to make sure the workers can receive and cooperate in realtime (Badges: Embedding Numbnuts, Go with the flow, Pug N’ Play, Power user love, Right Now, Stairway to heaven).

Business in the Front, Party in the Peach Mini Games: Where Fun Meets Fortune!

Posted on by

This post has the topic within Most Extreme Business Value category

Dataminer

Dash it out

At first glance, Peach Mini Games might seem like pure fun – a temporary escape from the realities of everyday life and an invitation to indulge into a dream. However, that’s where one truly misses the mark. When deeply exploring Peach Mini Games it is like stepping into a classic ‘Mullet’ situation: Business in the front, party in the back. Or maybe it is the other way around. Anyway…

You’re met with vibrant colors, pixel-perfection that would make anyone envious, and scalability smoother than a cat’s movement. But beneath this lies a complex tapestry of business value.

Hangman
Through our Hangman game, one not only learns the player meanings of words but also how they are spelled. Research has shown that the use of Hangman for these purposes is highly effective. Studies indicate that this method is motivational for learning, providing a playful approach that makes it more engaging for participants. Additionally, research suggests that interactivity, such as actively participating in game-based learning like Hangman, can enhance vocabulary and spelling skills more effectively than traditional methods.

Reference: https://jurnal.untan.ac.id/index.php/JEEP/article/viewFile/57-65/75676588091

Quiz
To complete the task of saving Mario, Princess Peach must pass a quiz. The quiz is integrated with ChatGPT and serves both as an entertaining and educational part of the game. At the same time, the concept is so versatile that it can be directly applied to create business value in the field of education.

Imagine a scenario where one configures a quiz within a specific subject area. Students can use this quiz to practice and identify knowledge gaps, while ensuring they rely on reliable sources. This setup provides teachers with control over information access, limiting students’ searches to a secure environment instead of allowing unrestricted internet browsing.

Statistics
Similar to how the games described above can motivate learning, our PowerBI scoreboard can better equip teachers to understand what students know and what they need to work on, both on a group and individual level. The statistics can pinpoint specific areas that require more attention, ensuring teachers use their time effectively and work in a more data-driven manner.

Data mine(kart)

Posted on by

In order to train our AI-model we need a lot data, but in order to have the correct data, we also need bananas. Being IT people, we never thought of using actual bananas, but instead had a hotel employee in a suit deliver 20 a3 pages of printed bananas that we cut out and glued to pieces of cardboard. Compete with their individual banana stands:

We then placed the bananas in the road around the hotel, got in the car and drove around filming the road from the dashboard inside the car. The conditions were quite foggy and bad, but the enthusiasm was electric. We ended up with a lot of footage including some of a consenting dog and owner.

We then started feeding all the videos to our model, mining for frames that contained actual bananas – our gold.

This is done in a jupyter notebook in python. The first thing the model does is identify all frames that contained potenital bananas, and singling them out. We ended up with over 10 000 potential banana frames. This is one of them with an obvious banana:

The model then strips away everything that is not a banana, in order to get the precise location of the banana:

Furthermore the model does a cutout of the frame around the banana which ends up being used in our actual dataset giving the banana detection app an excellent base and understanding of how to identify bananas.

Actual video from the training data:

Video of banana making:

Photo gallery of Herman and Erik placing bananas and driving around filming the bananas.

Data Miner

Posted on by

For our solution we heavily rely on 3.rd party services from AI to deliver a unique gaming experience. The first step in our application is asking questions to the user about the experience they want to play.

The values are sent over to a flow that has a custom connection to OpenAI prompting. Based on the promts this provides we will generate a unique gaming experience to our customers.

The first and most important promt is “Creating the Landskape”

The JSON that is received above is a complex structure of X and Y cordinate values. Each one representing a brick or movable space for the game.

Next step is evaluating this data in Power Apps, and the result is a playable game like the one below:

We have several more promts (Boxes in RED) that make up other elements of the game. This combines the API’s of DALL-E and OpenAI Chat promting. Together with these external API’s we generate a unique experience for the gamer each time they play. This value makes us one of the few games out there you can play time and time again without ever having to replay the same track!

Leveraging External Data to Enhance Mario Kart Race Simulations

Posted on by

We’re excited to announce a significant enhancement to our Mario Kart race simulations! By integrating data from mariowiki.com, we’ve enriched our simulations with detailed attributes for karts, drivers, tires, and gliders.

mariowiki

detailed stats

The data was easily available in table format already, so we just copied and pasted it into excel and uploaded it to dataverse. We decided to not make it more complicated than necessary

We then integrated the static data from mariowiki with out own python script to generate and simulate races. Using actual characters and maps enriched our racing statistics, and brings it closer to reality.

This image has an empty alt attribute; its file name is image-108-1024x420.png

PlumbBot is your first line of defence against a clogged toilet!

Posted on by

We built a web-based platform that lets you chat with a custom Copilot that knows everything about plumbing – PlumbBot. The Copilot is created with Copilot Studio and integrated in our Power Pages app as a part of our Plumber as a Service (PLaaS), and it rocks two of the hackathon badges: Crawler and Hipster.

Crawler: Using search in an awesome way

The Copilot has a huge knowledge base on plumbing. By providing domain knowledge of plumbing and practical plumbing solutions through files like PDFs or other text documents (as well as authenticated, private resources). By embedding the documents, Copilot uses vector search to find relevant paragraphs for grounding the Copilot, as well as providing helpful references for more in-depth details.

Here we have extended the knowledge base of by providing PDF documents containing domain knowledge and default high content moderation

Dataminer: strategic use of external data to enrich our solution’s business value
By enhancing our knowledge base with specialized PDF documents on plumbing, we’ve not just enriched our repository—we’ve transformed it. This approach not only solves practical business problems by making specialized knowledge readily accessible but also adds significant value to our existing data.

Moreover, the trainee can iteratively ask for details for each step for more detailed explanations and specifics regarding plumbing tools or plumbing terminology, directly on the Member’s homepage.

Here we have embedded PlumbBot in our Power Pages Member section. Proividing expert plumbing help to our authenticated mebers

Hipster: Using the hip tech in a safe way

To make sure we could trust the advice given by our PlumbBot, we only allow it to use Generative AI when triggered with highly specific plumbing terminology or words related to plumbing issues. Moreover dynamic chaining is disabled to limit any misleading information from other, unverified connections:

Specific trigger phrases
Only triggered in plumbing scenarios
Handling a malicious prompt

But most importantly, by default, Copilot locks high content moderation to uploaded files as shown above. This makes sure the Copilot is grounded when prompted and will not answer to request out of its scope and knowledge domain.

Moreover, when attempting to force the PlumbBot to give misleading answers or create unrealistic scenarios (e.g. not having a main water valve to the building) for plumbing issues, it continues to only use the grounding, domain specific documents provided for the PlumbBot. While iteratively attempting to mislead it, it remains true to the knowledge base due to the scoping of the bot.

As this is super hip tech, the MS Doc is slightly misleading.

It says you can create add a Custom Copilot in the new Power Pages studio, while the link refers to the legacy studio, as the new studio doesn’t properly support it:

But that doesn’t stop us from using bleeding edge technology for providing customer and business value, with user privacy!

The Bi-Pirates :)

Posted on by

Once upon a time, in the world of Business Intelligence, there was a ship with 3 crew members, who were struggling to make ends meet. They would sail from port to port, hoping to find some treasure, but to no avail. That was until they discovered Power BI.

They were able to analyze data from different sources and turn it into valuable insights. They used Power BI’s interactive visuals and dashboards to identify the ports and ships with the most valuable cargo. No more guessing, no more wandering aimlessly. so were able to sail straight to the ports or ships with the most valuable treasures. 🙂

Mobil view

used jason file to use “shape map” in power BI https://github.com/deldersveld/topojson

Naming the bastards!

Posted on by

Our solution is using Power Automate and HTTP trigger to accept new pirates

And put into a workflow used by our Pirate onboarding app.

Badges

Dataminer – The great database of LLM allows us to mine for awesome pirate names and visual traits for our crew.

Hipster – LLM are so hot, it will be a while until they are not. A multi-billion tech, via Azure Open AI studio cannot be beaten!

Existential Risk – It’s mining, it’s a huge learned model, living on the edge! Sure, we didn’t spend the billions training the model, but that’s hard to do on a pirate’s salary in 2023.