Greetings, magical technologists! ✨ At Team PowerPotters, we’ve combined the power of Bing Search API, OpenAI GPT-4, and Power Automate to create a truly innovative web-crawling solution. Our workflow dynamically discovers, analyzes, and integrates external APIs, transforming how potion-related data is sourced and utilized. Here’s how we earned the Crawler Badge by blending search, AI, and automation into one seamless process.
The Crawler Workflow: Step-by-Step Magic
Discovering APIs with Bing Search API
Purpose: To dynamically find public APIs related to Harry Potter spells or magical data.
Execution:
Query: "public API Harry Potter spells".
Filters: Results are restricted to recent entries with keywords like /api or “documentation”.
Analyzing APIs with OpenAI GPT-4
Purpose: To validate URLs as APIs and extract relevant schemas, field mappings, and example data.
Execution: For each URL, OpenAI determines if the URL links to an API or its documentation. If valid, it provides:
API schema.
Example JSON response.
Field mappings for key data (e.g., Name for spell name, Description for spell effects).
Integrating with Power Automate
Purpose: To process, validate, and integrate the data into our system.
Workflow Steps:
Parse Bing Results: Extract relevant URLs using JSON parsing.
Validate URLs: OpenAI determines if the URL links to a valid API and provides field mappings.
Dynamic Integration: Call validated APIs and use extracted data to:
Create new product entries in D365FO.
Enrich existing products with spell names (Name) and effects (Description).
Automation: Run schema validations dynamically, ensuring data consistency.
A Scenario in Action
A potion master requests information on new magical spells for potion research. Using this workflow:
Search: Bing Search API identifies APIs like Potterhead API.
Validation: OpenAI ensures the API provides valid spell data, extracting fields like Name (e.g., Accio) and Description (e.g., Summons an object).
Integration: Power Automate dynamically updates the potion master’s research database with enriched spell information, saving hours of manual effort.
Why This Deserves the Crawler Badge
Innovative Use of Search:
Bing Search API dynamically finds and filters public APIs, extending its use beyond static results.
AI-Powered Validation:
OpenAI GPT-4 dynamically analyzes URLs, validates APIs, and generates schemas and field mappings for seamless integration.
Solving Real Business Problems:
Potion masters gain enriched, real-time product data without manual intervention, enabling informed decisions.
Scalability:
The workflow is adaptable for future needs, such as integrating potions, artifacts, or even non-magical domains.
Crawling the Web for Magical Insights
This dynamic web-crawling solution exemplifies how search, AI, and automation can revolutionize the way data is discovered and integrated. With this innovation, we humbly submit our case for the Crawler Badge, showcasing how Team PowerPotters continues to push boundaries: acdc.blog/category/cepheo25.
The area of AI and Copilot has started and finally has reached the wizarding world. The Ministry of Magic is using several AI-based features to automate their Letter heavy processes.
And now it’s also time that school like Hogwarts use AI to support students and upcoming students with their application process.
This follows certain guidelines to protect the magical users privacy and the right handling of sensible data. We don’t want to spill the tea about death eater, unforgiving curses or you-know-who don’t we?
First of all, we want to protect the privacy of the users and internal business data. Such as the information about our students and their applications
This sensible data is excluded from our externally accessible HermioneAI. We only make that available as knowledge base for specific teachers on Microsoft Teams.
Additional the authentication itself is based on Microsoft Entra, to limit the risks of leaking sensible data.
This makes general and internal knowledge available to selected teachers of the school:
In this example, we use AI to search for specific information from Dataverse and display internal data that has been summarized and collected for easy access to the End User Wizard.
The handling of information and authentication is different on our web application:https://owlexpress.app/chat
As this is publicly available, mainly for students going through the application process, no authentication is necessary. Also these users don’t have access to internal data and only rely on Hermione’s general knowledge. Which already is huge.
Similar to the internal chat experience, the external relies on security measurements to avoid certain topics or harmful content. This for example covers Death eaters, You-Know-Who and the dark side of magic in general, but also jailbreak attempts
So, if a potential student or any other user of the chatbot on the website asks about joining the dark side, forbidden spells or details on You-know-who, we won’t help here
As a team, we have a primary goal: We help magic happen by utilizing modern AI approaches.
We also understand that every student needs a mentor, but the number of available mentors is minimal today. Our idea is to introduce digital twins for available mentors, which would allow us to help a more significant number of students. To achieve that, we are using all available data about the mentors. But to keep it fresh and reliable, we must have the latest events in our OneLake.
So, we implemented the LinkedIn profiles’ social crawlers to track all the mentors’ activities and events and suggest new potential mentors.
We do that via the most modern and most straightforward way by utilizing the PowerAutomate Desktop:
Then, the data will be collected and stored in an Excel spreadsheet, making it available for future processing with our Data Factory.
The primary consumer of the data regarding upcoming events is the students, who use the complex search request to find the proper event by searching across different fields. Participation in community events potentially increases the chances of successful onboarding to the new school.
Relevant search API functionality if covering 98% percent of the required functionality. Unfortunately, this API is unavailable on the Power Pages side. We implemented the Power Automate Flow from the portal, a wrapper around the original Dataverse API, to resolve that issue.
On the portal side, we are using the Select2 component to implement autocomplete functionality.
Recently updates including two more badges: Crawler and Dataminer.
Api, datasets and code are handy tools to share. But as a PowerBI rooky, it can just as difficult to get a overview and navigate through the magic Power BI provides. Thats why I wanted to share some insight into this area to make it easier for those who comes after us.
We have been struggeling for hours to find a way to embed the PowerBI report as an interactive report in the app. With a free trail not all features are possible, and some workarounds are needed.
After a whole lot of tries and fails as you can see below…
… we managed to put a PowerBI tile in the canvas app, with the wanted dimentions to match our apps design.
After publising your report to PowerBI service the same report can be saved as a tile in a dashboard. Having the whole report in one tile will make the vizualisations interactive with each other – which is more user-friendly.
Embedding a Dashboard tile in the Canvas app was not the biggest issue – making it mobile friendly was a bigger problem. It was not possible to create a dashboard with a mobile display that could be embedded in the app. However, splitting one Raport into 2 Dashboard tiles seemed to be the best way to get the wanted result (visually), even though it was not as interactive as we wanted. But its good enough – as shown below.
Getting the data
Dataminer: We created our own data in SharePoint lists showing the House Cup Points, combined with the other lists. This created the base for exporting this data directly into PowerBI
SharePoint as datasource
Lists: Students, PointSystem, Houses, Professors
In the semantic model we configuredSchedueled refresh (every 6th hour), ensuring that the data updates often enough so the students can keep track of their progress.
The semantic model provided enough data to create the golden report we later embedded in the app.
Crawler: Search and AI in PowerBI report
In PowerBI we included this cool visual where the data is searchable simply by using natural language to ask what the data contains. This is a flexible way for users to access data even though it is not in any of the visuals.
Let us demonstrate:
Helping out
Walking down the hallway – I bumped into Sopranova struggelig with the same thing I just figured out how to do. Lending a hand and providing some guidance we managed to create a custom page with a embedded dashboard tile for their app.
We were very happy about that.
So if anyone else needs a hand in this spesific area we are happy to help:)
The BPMN diagram below is a simple representation of our KoopaKonstruct Manager.
In summary we are using Hololenses to inspect a chosen area where we want to e.g. hang up a coat rack, build a table, and similar. In order to make the Hololenses smart, we have integrated OpenAI. We are using Function calling to provide instructions about the requested project, information about necessary materials and quantity.This is where we are making our KoopaKonstruct really smart.
As real #Crawlers we use OpenAI’s Function calling to dynamically create the search criterias based on natural language. Google Speach to text is used to capture the natural language query. We are really integrating into everything that flips bits #Embedding numbnuts! With help of the search criterias, we are able to provide critical information further into the supply chain, meaning all the way down to the FinOps where we are improving critical business processes (this will be explained later in this blog post).
Below is a picture from the Hololenses where we use a button to speak with AI:
GPTVision helps us interpret work drawings and breaks it down into the steps needed, so that we can get a clear instruction of what materials we need, how many and how to build the desired object. Using Power Automate, we are able to share and send the item details and necessary quantity into FinOps.
As an addition to the OpenAI magic, we have also created a canvas app that is integrated in the Hololense #PlugN’Play. From this app you can perform business processes, such as triggering sales orders, check inventory status, and 4 different Power BI reports to monitor sales orders and follow up late deliveries #Dash it out.
And here is where we really improve business processes; when the information is sent to FinOps we use X++ code to check inventory status. If materials are in stock, a sales order is created and sales order confirmation is sent. If there are no (or not enough) materials in stock, we also create a purchase order in addition to the sales order. A link between the purchase order and sales order is created, enabling the ability to understand where the demand is coming from. Additionally, to make the KoopaKonstruct Manager adaptable to e.g. construction companies, we have created the possibility to link the purchase and sales orders with projects. This provides a lot of possibilities for detailed project management for the finance geeks.
Below is our argumentation to why we believe we should score high on all 4 categories!
Most Extreme Business Value
Our team embarked on a quest to redefine the meaning of business value, and we present to you KoopaKonstruct Manager – a solution that not only orders products through Hololenses but catapults business efficiency into a realm beyond imagination.
Automating the Complete Supply Chain with Hololenses
We harnessed the power of Hololenses to revolutionize the way products are ordered. One user, equipped with the mighty Hololens, registers the necessary quantity with a mere wave of the hand, triggering a Power Automate flow like a seasoned Mario collecting power-ups.
The adventure doesn’t stop there! Our Power Automate flow seamlessly transmits the order information to FinOps using the Execute action. The magic unfolds as FinOps automatically generates a sales order and a purchase order based on inventory stock. But here’s where the plot thickens – these orders are not solitary entities; they are linked together, creating a dynamic duo that unravels the mystery behind each purchase. There is no need to spend as much as a second wondering how or why the orders were created; the link indicate that there is a demand behind each purchase order.
See yourself sitting in an ordering office at a company where the customers are able to generate sales orders by only entering a demand. And, in addition to the automatic creation of sales order, imagine that you did not have to study the master plan to generate the purchase order as it is automatically created! And as a side salsa, you did not even have to trigger the sales order confirmation as it is sent automatically. So, while you are sitting there wondering about what to do with your life (and how long it takes before the CEO finds out that you are useless as the power of Power Automate and X++ took your job), you can relax and focus on other more important things in life, such as playing a Super Mario game.
Power Stars: Automated Project Linkage
But wait, there’s more! We also increased the business value for construction and engineering enterprises. With the ability to automatically link both the purchase and sales orders to projects, we’ve bestowed upon industries that thrive on projects a golden opportunity to enhance their efficiency and project management. It’s not just about ordering; it’s about constructing a bridge to a new era of project-linked prosperity.
The Warp Whistle to Most Extreme Business Value
We make a bold claim for the “Most Extreme Business Value” category in this year’s ACDC Hackathon. Here’s why our adventure deserves the highest score:
Efficiency: Our solution isn’t just about ordering; it’s about efficiency power-ups that automate the entire supply chain process, from order initiation to linkage in FinOps.
Dynamic and Time-saving: The automatic creation of linked sales and purchase orders provides unparalleled visibility into demand and supply, laying the foundation for strategic business decisions.
Project Linkage: For construction and engineering businesses, our automated project linkage is a treasure trove of golden coins, offering enhanced project management capabilities and business value beyond the ordinary.
Adaptable: Our solution is not only limited to the B2B-market, but can also be used for the B2C-market. Making the app available to customers, they can create orders automatically and receive high quality service as the complete supply chain has become even more efficient.
Excellent User Experience
We have created a Super Mario inspired canvas app that is adapted into the Hololenses. In other words, our app transforms ordinary into extraordinary!
Features:
Killer AI
Whether you’re a seasoned AI veteran or just dipping your toes into the mushroom-filled waters of innovation, our solution is a testament to the power that AI holds in shaping the future.
KoopaKonstruct Manager is not just a game-changer; it’s a game-transformer. Through the mastery of machine learning, we’ve empowered our solution to evolve, adapt, and learn from the challenges it encounters. From predicting order quantities to optimizing supply chain processes in the real world (and in FinOps), our AI-driven approach is a power-up for efficiency and foresight.
In the quest for the Killer AI “crown”, KoopaKonstruct offers more than just automated processes. It also includes magic in form of predictive insights, providing a glimpse into the future of demand, supply, and project landscapes. Strategic decision-making becomes second nature as our AI-driven system transforms data into actionable intelligence.
A mind of its own
In the pursuit of the existential risk badge, our AI solution goes beyond mere execution—it thinks, learns, and adapts with a mind of its own. Through autonomous decision-making capabilities, it navigates the complexities of supply chain processes and provides responses not even a 20 year old is able to find without searching the web.
KoopaKonstruct isn’t just smarter than a 5th grader; it engages in intelligent conversations that transcend the limitations of a classroom. It doesn’t just provide answers; it grapples with FinOps and supply chain complexities, understands nuances, and offers insights that reflect a level of intelligence resembling a dynamic dialogue.
Why KoopaKonstruct Manager deserves the Killer AI “crown”
Innovation Beyond the Blocks: KoopaKonstruct isn’t confined to the traditional boundaries. It breaks through the blocks of conventional solutions, paving the way for a new era of AI-driven innovation.
Adaptability in Every Pixel: Our solution dynamically adapts to the ever-changing business environment, ensuring resilience and adaptability in every pixel of our solution.
Mind Mastery: Our AI isn’t just a tool; it’s a mastery that elevates KoopaKonstruct into a symphony of efficiency, intelligence, and strategic foresight.
Pandoras Box
Equipping a Hololens with AI that can answer questions, order materials, and assist in selecting the right materials and tools for any project based on online instructions transforms DIY into an accessible experience for the less handy. This technology takes the guesswork out of building, offering tailored advice on what to build, the best materials and tools for the job, and even suggesting projects that match your interests and available space. It’s like having a personal tutor and assistant rolled into one, guiding you through the entire process from conception to completion. This seamless integration of technology with hands-on work not only makes crafting projects more approachable for beginners but also enriches the experience with educational value, ensuring that anyone can undertake and succeed in their DIY endeavors with confidence.
PromptFlow, our innovative AI chatbot integrated into Microsoft Teams, not only streamlines race strategy and collaboration but also stands as a testament to advanced cloud technology integration. Targeting three distinguished badges – “Plug N’ Play,” “Crawler,” and “Stairway To Heaven,” PromptFlow embodies the pinnacle of AI-driven solutions in Microsoft’s ecosystem.
Plug N’ Play: Enhancing Microsoft Teams with AI
PromptFlow elevates Microsoft Teams by offering automated, data-driven insights for racing strategy development. By utilizing over 5,000 race statistics, it enables users to query lap times, kart performance, and player stats in natural language, demonstrating a perfect blend of AI and user experience.
Crawler: Transforming Race Strategy with AI Search
The “Crawler” badge highlights PromptFlow’s innovative use of AI search to navigate extensive race data, revolutionizing the way racing teams strategize. This feature addresses the critical need for quick and accurate decision-making, offering a significant competitive edge in the racing industry.
Stairway To Heaven: A Symphony of Microsoft Cloud APIs
Achieving the “Stairway To Heaven” badge, PromptFlow masterfully combines multiple Microsoft cloud APIs:
AI Search: Powers the core functionality, enabling efficient data retrieval.
Sentiment Analysis: Enhances user interaction by adapting responses based on detected sentiment.
Azure OpenAI Instance: The backbone providing computational power and advanced LLM capabilities.
This integration not only meets the badge’s criteria but also sets a new standard for AI solutions in cloud environments.
Technical Integration and Future Prospects
Each API plays a crucial role in making PromptFlow a robust, intelligent system. This multifaceted integration exemplifies our commitment to leveraging cloud technology for creating advanced AI solutions, paving the way for future innovations.
The solution/demo:
Conclusion
PromptFlow is more than just a chatbot; it’s a groundbreaking tool that revolutionizes race strategy development and team collaboration. It showcases our expertise in blending AI with cloud technologies, earning us the “Plug N’ Play,” “Crawler,” and “Stairway To Heaven” badges. As we continue to innovate, we look forward to exploring new horizons in AI and cloud computing.
Note: This combined draft succinctly captures the essence of PromptFlow’s achievements in earning the three badges. It emphasizes the chatbot’s functionalities, technical prowess, and future potential in the realm of AI and cloud computing.
As the ACDC Hackathon draws to its climax, our team’s dedication and innovative spirit have led us to target three advanced badges, each reflecting our commitment to solving complex problems with cutting-edge technology and best practices.
Crawler: Revolutionizing Assistance with Copilot Studio
In our quest for the “Crawler” badge, we’ve integrated Copilot Studio into our Power Page site, embedding a Copilot bot designed to aid users when they’re stuck on a course level. This ingenious feature utilizes a cloud flow triggered by the user’s input, leveraging the Dataverse action “Search Rows (Preview)” to find and return answers for the specified course level. This not only enhances user experience but also showcases our innovative use of search to solve a real-world business problem.
Client Side Salsa: Ensuring Smooth Performance with Modern Frameworks
Our pursuit of the “Client Side Salsa” badge highlights our application’s fluidity, achieved by running entirely within the client’s browser without becoming a cumbersome monolith. By crafting a PCF control for our interactive course game using TypeScript and React, we ensure our setup’s compatibility across a wide range of browsers. This approach underlines our strategic use of modern front-end frameworks to deliver a seamless and responsive user interface.
ACDC Craftsman: Demonstrating Development and Deployment Excellence
The “ACDC Craftsman” badge is a testament to our adherence to development and deployment best practices. We’ve established three environments (dev, UAT, & prod), focusing our component development within the dev environment. Our deployment strategy is streamlined through Power Platform ALM pipelines, incorporating a pre-deployment approval step via Teams. This allows for meticulous control over deployment timing and participants. Moreover, our use of Azure DevOps Repositories for the PCF control exemplifies our commitment to source control and project management excellence.
As we conclude this hackathon journey, these badges represent more than just our technical achievements; they signify our team’s holistic approach to innovation, user engagement, and operational efficiency. Our efforts to blend advanced technological solutions with practical application and meticulous project management illustrate the essence of true ACDC craftsmanship, setting a high bar for what can be accomplished in such a collaborative and competitive environment.
For a innovative search method to find the game rules and instructions.
As we have developed a game for others to play, where we have used Copilot for the user to choose username, character and difficulty level. In addition to give the player the possibilities to understand the game beforehand we have made it possible for the player to ask the Copilot questions about the game and how it works.
We did so by uploading a PDF file to the Copilot with game instructions and a description of the game.
Since the Crawler badge is all about using a innovative way to search, we have used the generative answers for Copilot. This means that the player can use natural language when searching when asking about the game. An example is showcased in the photo below. The player are using the word “Intention”, while the copilot is answering by using “purpose”. This means the players can use natural language when asking about the game.
We are implementing a face recognition app, Our app can address several challenges and provide solutions to various problems. Face recognition can improve security by replacing or supplementing traditional access control methods and face recognition can provide a convenient and secure method for verifying identities.
To make our amazing service Tubi work, a lot of cloud is needed. We aim to make the plumber’s job easier by recommending the best layout for where the pipes should go, and for that, we need AI. We have trained a model in Custom Vision to recognize all components in a bathroom that need water. So, when the plumber uploads a floor plan to our Static Web App, the image is sent to our Azure Function App backend in C# Asp.net through our own API. But both the image and the equipment list must be stored somewhere. Therefore, we have also connected to Azure Blob Storage. Then last but not at all least. The people working in the back office have instant interactive reports available to help them with filing and billing through Power BI and alerting the using an automated flow (Badges: Feature Bombing)
Sometimes it works, and that’s plenty
Databases are good, but sometimes it’s easier to just dump everything in one place until you need it again. Yes, it might not be very scalable or very normalized. SQL became too heavy, and we already needed a Blob storage to store the images, so we also dump the order data in the same blob storage as JSON files. It’s old fashioned way of serverstorage, and a bit dirty, but it works! (Badges: Nasty hacker, Retro badge)
Power the backoffice
As the final list of components are decided, they still have to be approved from the accounting team in the office. To make sure they have all the information they require, we have developed a Power BI dashboard to crawl through our registered data and make sure the orders are handled properly (Badges: Crawler, Dash it Out, Dataminer). And to make sure the orders are handled easy and fast, the dashboard is embedded into teams and an alert is automated by using a logic app to make sure the workers can receive and cooperate in realtime (Badges: Embedding Numbnuts, Go with the flow, Pug N’ Play, Power user love, Right Now, Stairway to heaven).
