Tag Archives: Crawler

Final Delivery

Posted on by

The Final Solution: KoopaKonstruct Manager

The BPMN diagram below is a simple representation of our KoopaKonstruct Manager.

In summary we are using Hololenses to inspect a chosen area where we want to e.g. hang up a coat rack, build a table, and similar. In order to make the Hololenses smart, we have integrated OpenAI. We are using Function calling to provide instructions about the requested project, information about necessary materials and quantity.This is where we are making our KoopaKonstruct really smart.

As real #Crawlers we use OpenAI’s Function calling to dynamically create the search criterias based on natural language. Google Speach to text is used to capture the natural language query. We are really integrating into everything that flips bits #Embedding numbnuts! With help of the search criterias, we are able to provide critical information further into the supply chain, meaning all the way down to the FinOps where we are improving critical business processes (this will be explained later in this blog post).

Below is a picture from the Hololenses where we use a button to speak with AI:

GPTVision helps us interpret work drawings and breaks it down into the steps needed, so that we can get a clear instruction of what materials we need, how many and how to build the desired object. Using Power Automate, we are able to share and send the item details and necessary quantity into FinOps.

As an addition to the OpenAI magic, we have also created a canvas app that is integrated in the Hololense #PlugN’Play. From this app you can perform business processes, such as triggering sales orders, check inventory status, and 4 different Power BI reports to monitor sales orders and follow up late deliveries #Dash it out.

And here is where we really improve business processes; when the information is sent to FinOps we use X++ code to check inventory status. If materials are in stock, a sales order is created and sales order confirmation is sent. If there are no (or not enough) materials in stock, we also create a purchase order in addition to the sales order. A link between the purchase order and sales order is created, enabling the ability to understand where the demand is coming from. Additionally, to make the KoopaKonstruct Manager adaptable to e.g. construction companies, we have created the possibility to link the purchase and sales orders with projects. This provides a lot of possibilities for detailed project management for the finance geeks.

Below is our argumentation to why we believe we should score high on all 4 categories!

Most Extreme Business Value

Our team embarked on a quest to redefine the meaning of business value, and we present to you KoopaKonstruct Manager – a solution that not only orders products through Hololenses but catapults business efficiency into a realm beyond imagination.

Automating the Complete Supply Chain with Hololenses

We harnessed the power of Hololenses to revolutionize the way products are ordered. One user, equipped with the mighty Hololens, registers the necessary quantity with a mere wave of the hand, triggering a Power Automate flow like a seasoned Mario collecting power-ups.

The adventure doesn’t stop there! Our Power Automate flow seamlessly transmits the order information to FinOps using the Execute action. The magic unfolds as FinOps automatically generates a sales order and a purchase order based on inventory stock. But here’s where the plot thickens – these orders are not solitary entities; they are linked together, creating a dynamic duo that unravels the mystery behind each purchase. There is no need to spend as much as a second wondering how or why the orders were created; the link indicate that there is a demand behind each purchase order.

See yourself sitting in an ordering office at a company where the customers are able to generate sales orders by only entering a demand. And, in addition to the automatic creation of sales order, imagine that you did not have to study the master plan to generate the purchase order as it is automatically created! And as a side salsa, you did not even have to trigger the sales order confirmation as it is sent automatically. So, while you are sitting there wondering about what to do with your life (and how long it takes before the CEO finds out that you are useless as the power of Power Automate and X++ took your job), you can relax and focus on other more important things in life, such as playing a Super Mario game.

Power Stars: Automated Project Linkage

But wait, there’s more! We also increased the business value for construction and engineering enterprises. With the ability to automatically link both the purchase and sales orders to projects, we’ve bestowed upon industries that thrive on projects a golden opportunity to enhance their efficiency and project management. It’s not just about ordering; it’s about constructing a bridge to a new era of project-linked prosperity.

The Warp Whistle to Most Extreme Business Value

We make a bold claim for the “Most Extreme Business Value” category in this year’s ACDC Hackathon. Here’s why our adventure deserves the highest score:

  1. Efficiency: Our solution isn’t just about ordering; it’s about efficiency power-ups that automate the entire supply chain process, from order initiation to linkage in FinOps.
  2. Dynamic and Time-saving: The automatic creation of linked sales and purchase orders provides unparalleled visibility into demand and supply, laying the foundation for strategic business decisions.
  3. Project Linkage: For construction and engineering businesses, our automated project linkage is a treasure trove of golden coins, offering enhanced project management capabilities and business value beyond the ordinary.
  4. Adaptable: Our solution is not only limited to the B2B-market, but can also be used for the B2C-market. Making the app available to customers, they can create orders automatically and receive high quality service as the complete supply chain has become even more efficient.

Excellent User Experience

We have created a Super Mario inspired canvas app that is adapted into the Hololenses. In other words, our app transforms ordinary into extraordinary!

Features:

Killer AI

Whether you’re a seasoned AI veteran or just dipping your toes into the mushroom-filled waters of innovation, our solution is a testament to the power that AI holds in shaping the future.

KoopaKonstruct Manager is not just a game-changer; it’s a game-transformer. Through the mastery of machine learning, we’ve empowered our solution to evolve, adapt, and learn from the challenges it encounters. From predicting order quantities to optimizing supply chain processes in the real world (and in FinOps), our AI-driven approach is a power-up for efficiency and foresight.

In the quest for the Killer AI “crown”, KoopaKonstruct offers more than just automated processes. It also includes magic in form of predictive insights, providing a glimpse into the future of demand, supply, and project landscapes. Strategic decision-making becomes second nature as our AI-driven system transforms data into actionable intelligence.

A mind of its own

In the pursuit of the existential risk badge, our AI solution goes beyond mere execution—it thinks, learns, and adapts with a mind of its own. Through autonomous decision-making capabilities, it navigates the complexities of supply chain processes and provides responses not even a 20 year old is able to find without searching the web.

KoopaKonstruct isn’t just smarter than a 5th grader; it engages in intelligent conversations that transcend the limitations of a classroom. It doesn’t just provide answers; it grapples with FinOps and supply chain complexities, understands nuances, and offers insights that reflect a level of intelligence resembling a dynamic dialogue.

Why KoopaKonstruct Manager deserves the Killer AI “crown”

  1. Innovation Beyond the Blocks: KoopaKonstruct isn’t confined to the traditional boundaries. It breaks through the blocks of conventional solutions, paving the way for a new era of AI-driven innovation.
  2. Adaptability in Every Pixel: Our solution dynamically adapts to the ever-changing business environment, ensuring resilience and adaptability in every pixel of our solution.
  3. Mind Mastery: Our AI isn’t just a tool; it’s a mastery that elevates KoopaKonstruct into a symphony of efficiency, intelligence, and strategic foresight.

Pandoras Box

Equipping a Hololens with AI that can answer questions, order materials, and assist in selecting the right materials and tools for any project based on online instructions transforms DIY into an accessible experience for the less handy. This technology takes the guesswork out of building, offering tailored advice on what to build, the best materials and tools for the job, and even suggesting projects that match your interests and available space. It’s like having a personal tutor and assistant rolled into one, guiding you through the entire process from conception to completion. This seamless integration of technology with hands-on work not only makes crafting projects more approachable for beginners but also enriches the experience with educational value, ensuring that anyone can undertake and succeed in their DIY endeavors with confidence.

PromptFlow: A Multifaceted AI Solution in Microsoft Teams 

Posted on by

Introduction 

PromptFlow, our innovative AI chatbot integrated into Microsoft Teams, not only streamlines race strategy and collaboration but also stands as a testament to advanced cloud technology integration. Targeting three distinguished badges – “Plug N’ Play,” “Crawler,” and “Stairway To Heaven,” PromptFlow embodies the pinnacle of AI-driven solutions in Microsoft’s ecosystem. 

Plug N’ Play: Enhancing Microsoft Teams with AI 

PromptFlow elevates Microsoft Teams by offering automated, data-driven insights for racing strategy development. By utilizing over 5,000 race statistics, it enables users to query lap times, kart performance, and player stats in natural language, demonstrating a perfect blend of AI and user experience. 

Crawler: Transforming Race Strategy with AI Search 

The “Crawler” badge highlights PromptFlow’s innovative use of AI search to navigate extensive race data, revolutionizing the way racing teams strategize. This feature addresses the critical need for quick and accurate decision-making, offering a significant competitive edge in the racing industry. 

Stairway To Heaven: A Symphony of Microsoft Cloud APIs 

Achieving the “Stairway To Heaven” badge, PromptFlow masterfully combines multiple Microsoft cloud APIs: 

  • AI Search: Powers the core functionality, enabling efficient data retrieval. 
  • Sentiment Analysis: Enhances user interaction by adapting responses based on detected sentiment. 
  • PromptFlow Technology: Our proprietary tech, ensuring smooth and natural conversational experiences. 
  • Azure OpenAI Instance: The backbone providing computational power and advanced LLM capabilities. 

This integration not only meets the badge’s criteria but also sets a new standard for AI solutions in cloud environments. 

Technical Integration and Future Prospects 

Each API plays a crucial role in making PromptFlow a robust, intelligent system. This multifaceted integration exemplifies our commitment to leveraging cloud technology for creating advanced AI solutions, paving the way for future innovations. 

The solution/demo:

Conclusion 

PromptFlow is more than just a chatbot; it’s a groundbreaking tool that revolutionizes race strategy development and team collaboration. It showcases our expertise in blending AI with cloud technologies, earning us the “Plug N’ Play,” “Crawler,” and “Stairway To Heaven” badges. As we continue to innovate, we look forward to exploring new horizons in AI and cloud computing. 

Note: This combined draft succinctly captures the essence of PromptFlow’s achievements in earning the three badges. It emphasizes the chatbot’s functionalities, technical prowess, and future potential in the realm of AI and cloud computing. 

Final Stretch: Unlocking Advanced Badges on the Last Day of Development

Posted on by

As the ACDC Hackathon draws to its climax, our team’s dedication and innovative spirit have led us to target three advanced badges, each reflecting our commitment to solving complex problems with cutting-edge technology and best practices.

Crawler: Revolutionizing Assistance with Copilot Studio

In our quest for the “Crawler” badge, we’ve integrated Copilot Studio into our Power Page site, embedding a Copilot bot designed to aid users when they’re stuck on a course level. This ingenious feature utilizes a cloud flow triggered by the user’s input, leveraging the Dataverse action “Search Rows (Preview)” to find and return answers for the specified course level. This not only enhances user experience but also showcases our innovative use of search to solve a real-world business problem.

Client Side Salsa: Ensuring Smooth Performance with Modern Frameworks

Our pursuit of the “Client Side Salsa” badge highlights our application’s fluidity, achieved by running entirely within the client’s browser without becoming a cumbersome monolith. By crafting a PCF control for our interactive course game using TypeScript and React, we ensure our setup’s compatibility across a wide range of browsers. This approach underlines our strategic use of modern front-end frameworks to deliver a seamless and responsive user interface.

ACDC Craftsman: Demonstrating Development and Deployment Excellence

The “ACDC Craftsman” badge is a testament to our adherence to development and deployment best practices. We’ve established three environments (dev, UAT, & prod), focusing our component development within the dev environment. Our deployment strategy is streamlined through Power Platform ALM pipelines, incorporating a pre-deployment approval step via Teams. This allows for meticulous control over deployment timing and participants. Moreover, our use of Azure DevOps Repositories for the PCF control exemplifies our commitment to source control and project management excellence.

As we conclude this hackathon journey, these badges represent more than just our technical achievements; they signify our team’s holistic approach to innovation, user engagement, and operational efficiency. Our efforts to blend advanced technological solutions with practical application and meticulous project management illustrate the essence of true ACDC craftsmanship, setting a high bar for what can be accomplished in such a collaborative and competitive environment.

Crawler

Posted on by

For a innovative search method to find the game rules and instructions.

As we have developed a game for others to play, where we have used Copilot for the user to choose username, character and difficulty level. In addition to give the player the possibilities to understand the game beforehand we have made it possible for the player to ask the Copilot questions about the game and how it works.

We did so by uploading a PDF file to the Copilot with game instructions and a description of the game.

Since the Crawler badge is all about using a innovative way to search, we have used the generative answers for Copilot. This means that the player can use natural language when searching when asking about the game.
An example is showcased in the photo below. The player are using the word “Intention”, while the copilot is answering by using “purpose”. This means the players can use natural language when asking about the game.

Crawler

Posted on by

We are implementing a face recognition app, Our app can address several challenges and provide solutions to various problems. Face recognition can improve security by replacing or supplementing traditional access control methods and face recognition can provide a convenient and secure method for verifying identities.

Piping our way through the Azure Cloud

Posted on by

To make our amazing service Tubi work, a lot of cloud is needed. We aim to make the plumber’s job easier by recommending the best layout for where the pipes should go, and for that, we need AI. We have trained a model in Custom Vision to recognize all components in a bathroom that need water. So, when the plumber uploads a floor plan to our Static Web App, the image is sent to our Azure Function App backend in C# Asp.net through our own API. But both the image and the equipment list must be stored somewhere. Therefore, we have also connected to Azure Blob Storage. Then last but not at all least. The people working in the back office have instant interactive reports available to help them with filing and billing through Power BI and alerting the using an automated flow (Badges: Feature Bombing)

Sometimes it works, and that’s plenty

Databases are good, but sometimes it’s easier to just dump everything in one place until you need it again. Yes, it might not be very scalable or very normalized. SQL became too heavy, and we already needed a Blob storage to store the images, so we also dump the order data in the same blob storage as JSON files. It’s old fashioned way of serverstorage, and a bit dirty, but it works! (Badges: Nasty hacker, Retro badge)

Power the backoffice

As the final list of components are decided, they still have to be approved from the accounting team in the office. To make sure they have all the information they require, we have developed a Power BI dashboard to crawl through our registered data and make sure the orders are handled properly (Badges: Crawler, Dash it Out, Dataminer). And to make sure the orders are handled easy and fast, the dashboard is embedded into teams and an alert is automated by using a logic app to make sure the workers can receive and cooperate in realtime (Badges: Embedding Numbnuts, Go with the flow, Pug N’ Play, Power user love, Right Now, Stairway to heaven).

Find the right Mario for you

Posted on by

We claim badges:

  • Plug N’ Play – for Dynamics 365 plugin that updates Mario for each support request based on the context using OpenAI
  • The Existential Risk – for AI decision making and work assignment without human involvment
  • Go With The Flow – for automating of the technician assignment
  • Crawler – for an innovative approach of searching for the right Mario technician

Sometimes Princesses are confused and don’t know where to start from when they have technical issues. So we are here to help them with AI! Princesses just need to add the description of the problem and the Dynamics Plugin will suggest Marios that are currently available and best match the case.

So the Princess creates a new support request, and that’s were plugin is triggered. The plugin sends the problem to OpenAI’s smart ChatGPT. The prompt:
Given the following case description, rate its severity on a scale of 1 to 4, where 1 is minor and 4 is critical:

I seem to be having a bit of a predicament with our kingdom’s email system, and I’m in dire need of assistance. This morning, when I attempted to send out invitations for the upcoming royal ball, I noticed that none of my emails were being delivered. They seem to be stuck in the outbox, and no matter how many times I try to resend them, they simply won’t go through.

The result of the OpenAI request is the severity level, in our case it is 3. So the plugin is looking for Marios who can cope with this severity and assigns the case to this Mario. Also, plugin updates the details of the case:

It is an extraordinary way to search for an engineer, but also a risky for administrators of the support team, as AI is taking away their job! On the other hand – it follows the flow and automate the routine process, which is a major benefit for the business itself….

Five badges in one poost?

Posted on by

Toad, a character widely recognized as Mario’s loyal companion, assumes a different role in our scenario. Instead of assisting Mario, Toad will be guiding Princess Peach through the perilous landscapes of the Mushroom Kingdom. Known for his extensive knowledge, Toad has documented everything in his personal wiki, which serves as a comprehensive guide to their world. In our application, Toad will utilize this knowledge to answer Princess Peach’s queries via a chat interface. This interactive feature will allow Princess Peach to navigate her journey with greater confidence and understanding.

Toad, an AI bot powered by Copilot Studio, learns from the web and adapts to new data. It uses advanced machine learning algorithms to understand and interpret information. This continuous learning makes Toad a versatile tool for tasks like data analysis, content creation, and problem-solving. It represents the next generation of AI, ready to assist with its ever-growing knowledge. (New text added here)

Live demo of Toad:

Who can resist the thrill of a captivating game? Our app takes you on an exhilarating journey, brimming with elements of gamification. It features a sophisticated scoring system for our beloved Princess Peach, tracking her progress throughout the game.

Live demo of score:

But that’s not all! Princess Peach has access to a dynamic Power BI dashboard, allowing her to review her progress and identify her strengths and areas for improvement. This interactive tool serves as a constant motivator, pushing her to strive for excellence

Last night, we celebrated a significant achievement as we won the award for ‘Excellent User Experience’ for Thursday. This accolade is a testament to the hard work of our experienced UX designer, who has crafted a fabulous Figma prototype to assist our developers. This victory leads us to the final chapter of our journey.

Last but not least, we invite you to step into the world of our application. Here, a vibrant pink palette, reminiscent of Peach’s dress, meets a shiny and glossy user interface, crowned with a touch of yellow, much like Peach’s crown. Our design choices, carefully considered for ease of navigation, make interacting with our app not just user-friendly, but a truly delightful experience. (New text added here)

Color palette from Figma

Live demo of shiny and glossy design (the design is not 100% complete in this video):

This post was edited 03.02 12:40

New text is marked with: (New text added here)

  • Added live demo of bot and some text
  • Added live demo of gamification
  • Added live demo of glossy design, picture and some text

PlumbBot is your first line of defence against a clogged toilet!

Posted on by

We built a web-based platform that lets you chat with a custom Copilot that knows everything about plumbing – PlumbBot. The Copilot is created with Copilot Studio and integrated in our Power Pages app as a part of our Plumber as a Service (PLaaS), and it rocks two of the hackathon badges: Crawler and Hipster.

Crawler: Using search in an awesome way

The Copilot has a huge knowledge base on plumbing. By providing domain knowledge of plumbing and practical plumbing solutions through files like PDFs or other text documents (as well as authenticated, private resources). By embedding the documents, Copilot uses vector search to find relevant paragraphs for grounding the Copilot, as well as providing helpful references for more in-depth details.

Here we have extended the knowledge base of by providing PDF documents containing domain knowledge and default high content moderation

Dataminer: strategic use of external data to enrich our solution’s business value
By enhancing our knowledge base with specialized PDF documents on plumbing, we’ve not just enriched our repository—we’ve transformed it. This approach not only solves practical business problems by making specialized knowledge readily accessible but also adds significant value to our existing data.

Moreover, the trainee can iteratively ask for details for each step for more detailed explanations and specifics regarding plumbing tools or plumbing terminology, directly on the Member’s homepage.

Here we have embedded PlumbBot in our Power Pages Member section. Proividing expert plumbing help to our authenticated mebers

Hipster: Using the hip tech in a safe way

To make sure we could trust the advice given by our PlumbBot, we only allow it to use Generative AI when triggered with highly specific plumbing terminology or words related to plumbing issues. Moreover dynamic chaining is disabled to limit any misleading information from other, unverified connections:

Specific trigger phrases
Only triggered in plumbing scenarios
Handling a malicious prompt

But most importantly, by default, Copilot locks high content moderation to uploaded files as shown above. This makes sure the Copilot is grounded when prompted and will not answer to request out of its scope and knowledge domain.

Moreover, when attempting to force the PlumbBot to give misleading answers or create unrealistic scenarios (e.g. not having a main water valve to the building) for plumbing issues, it continues to only use the grounding, domain specific documents provided for the PlumbBot. While iteratively attempting to mislead it, it remains true to the knowledge base due to the scoping of the bot.

As this is super hip tech, the MS Doc is slightly misleading.

It says you can create add a Custom Copilot in the new Power Pages studio, while the link refers to the legacy studio, as the new studio doesn’t properly support it:

But that doesn’t stop us from using bleeding edge technology for providing customer and business value, with user privacy!