We already have access to MR (aka. Magic Reality) components in Canvas Apps. Implementation is straight forward, but as expected they come with a limited set of features. The components are based on Babylon.js, and make for a quick and easy way to place and view a 3D model in an augmented reality context.
For our solution, we wanted the user to also be able to interact with the virtual objects in front of them, which is not OOB features, so by expressing our power user love, we decided to explore the possibilities around custom XR enabled PCF components.
Being ACDC craftsmen, knowing the potential issues of going too far down the wrong path, we decided to do some proof of concepts, creating custom PCF components with third party XR libraries, acting like proper thieving bastards on the way.
First off, we had a look at AR.js, which is built on ARToolkit, a relatively old library. This library could provide us with wide device support, which really didn’t have that much value, considering the component would be running inside the Power Apps mobile app. We would also be forced to use either image target or marker tracking, with no modern AR spatial tracking.
Looking closer at the OOB components, we tried to find a way to leverage the OOB Babylon.js logic, hopefully being able to hook into the React Native part of the implementation, which would give great benefits in terms of access to device specific XR features (ARCore for Android and ARKit for iOS). We did, however, decide to leave this path, and focus elsewhere.
In our solution, users will be gathering ingredients using object detection in a Canvas App. The AI model used for this has been trained on objects around the conference venue, and so we wanted to enhance the connection between the app and the real world. Already having access to the users geo location through the geolocation web API inside the Canvas App and any PCF components, we decided to these data to place the active users on a 3D representation of the venue, expressing our power user love by merging 3D graphics with the OOB Canvas App elements.
We were able to find a simple volume model of the buildings on the map service Kommunekart 3D, but these data seem to be provided by Norkart, which is not freely available.
Like the thieving bastards we are, we decided to scrape the 3D model off of the site, by fetching all the resources that looked like binary 3D data. We found the data was in B3DM format and we found the buildings in one of these. We used Blender to clean up the model, by removing surrounding buildings and exporting it to glTF 3D file format, for use in a WebGL 3D context.
The representation of the 3D model, we decided to do with Three.js, which let us create an HTML canvas element inside the PCF component and using its WebGL context to render out the model in 3D. The canvas is continuously rendered using requestAnimationFrame under the hood, making it efficient in a browser context. The glTF model was loaded using a data URI, as a workaround for the web resource file format restrictions.
The coordinates from the user’s mobile device comes in as geographical coordinates, with longitude, latitude and altitude. The next step was to map these values relative to a known coordinate in the building, which we chose to be the main entrance. By using the main entrance geographical coordinates, we could then convert that to cartesian coordinates, with X, Y and Z, do the same to the realtime coordinates from the user, and subtract the origin, to get the offset in meters. The conversion from geographic to geocentric coordinates were done like so:
// eslint-disable-next-line @typescript-eslint/consistent-type-definitions
export type CartesianCoordinates = { x: number; y: number; z: number };
// eslint-disable-next-line @typescript-eslint/consistent-type-definitions
export type GeographicCoordinates = { lat: number; lon: number; alt: number };
// Conversion factor from degrees to radians
const DEG_TO_RAD = Math.PI / 180;
// Constants for WGS84 Ellipsoid
const WGS84_A = 6378137.0; // Semi-major axis in meters
const WGS84_E2 = 0.00669437999014; // Square of eccentricity
// Function to convert geographic coordinates (lat, lon, alt) to ECEF (x, y, z)
export function geographicToECEF(coords: GeographicCoordinates): { x: number; y: number; z: number } {
// Convert degrees to radians
const latRad = coords.lat * DEG_TO_RAD;
const lonRad = coords.lon * DEG_TO_RAD;
// Calculate the radius of curvature in the prime vertical
const N = WGS84_A / Math.sqrt(1 - WGS84_E2 * Math.sin(latRad) * Math.sin(latRad));
// ECEF coordinates
const x = (N + coords.alt) * Math.cos(latRad) * Math.cos(lonRad);
const y = (N + coords.alt) * Math.cos(latRad) * Math.sin(lonRad);
const z = (N * (1 - WGS84_E2) + coords.alt) * Math.sin(latRad);
return { x, y, z };
}
This gave us fairly good precision, but not without the expected inaccuracy caused by being indoors.
In our solution the current position is then represented by an icon moving around the 3D model based on the current GPS data from the device.
To connect this representation to realtime data from all the currently active users, we decided to set up an Azure SignalR Service, with an accompanying Azure Storage and Azure Function App for the backend, bringing it all to the cloud, almost like a stairway to heaven. With this setup, we could use the @microsoft/azure package inside the PCF component, receiving connection, disconnection and location update message broadcast from all other users, showing where they are right now.
In Hogwarts school of witchcraft and wizardry, everything is about getting points. And who wouldn’t a pet that could give (or take away) points?
So, by using our datadriven Power app, we made this an option and made sure it would sync between Dataverse, Fabric and display them in the mirror by retriving the data using javascript.
In our app we have made use of the great opensource libraries of ready-made opportunities through the PowerApps Component Framework (PCF) Gallery. Inspired by the playful spirit of Fred and George Weasley from Harry Potter, these components add a touch of magic to the app: “Shaking Text” and “Confetti.”
Navigating all the lectures at Hogwarts can be tricky, but fear not! We’ve got a Time-Turner feature, just like Hermione’s, to ensure you never miss a single lesson. With this magical tool, you’ll be able to attend every class and ace your studies! 🕰️✨
Fred and George Weasley are known for their pranks and infectious energy. The “Shaking Text” component adds a dynamic, eye-catching effect to text, while the “Confetti” component brings a celebratory vibe, both reflecting the twins’ lively personalities. Power Apps
The Confetti component and the Shaking text component is published by Clavin Fernandes. They are built using TypeScript and React, and the Confetti component utilizes an NPM package. The developer on the team reviewed the source code and checked dependencies before importing the components into our solution. This thorough review ensures that all dependencies are compatible and up-to-date, preventing potential conflicts and integration issues. By doing this, the developer helps maintain the stability and reliability of the solution, ensuring a smooth and error-free deployment.
The PCF components are built using TypeScript and React, allowing for custom functionality and appearance in Power Apps. They integrate seamlessly and are reusable across different applications, enhancing the user experience. By leveraging these technologies, developers can create highly interactive and visually appealing components that meet specific business needs.
Say hello to DobbAI, our AI-powered assistant with the charm and wit of a real house-elf, ready to serve your workplace needs. From fetching information to assisting with complex tasks, DobbAI is designed to make data as accessible as asking a simple question.
DobbAI exemplifies #ProCodeNoCodeUnite by blending Microsoft’s Copilot Studio, Power Platform, and React to create a seamless, engaging, and efficient experience.
What is DobbAI?
DobbAI is more than just an AI tool—it’s a Talk-to-Your-Data CoPilot, seamlessly integrated into our workplace ecosystem.
Here’s what makes it special:
Charming and Knowledgeable: Like a helpful house-elf, DobbAI delivers witty, accurate responses to queries.
Deeply Integrated: It’s fully connected to the intranet, housed on SharePoint, so it can fetch everything from the dinner menu in the Great Hall, Class Schedules, and Academic Calender Information, to the standings of the House Cup (pulled from a SharePoint list, of course).
Effortless Access: The AI is published to a custom website channel, providing an iframe that integrates flawlessly with our React PCF (PowerApps Component Framework) component, meticulously deployed to the Power App solution.
Whether you need a quick answer, task automation, or AI-driven insights, DobbAI has you covered.
Low-Code Meets Pro-Code: How It Works
DobbAI demonstrates how low-code platforms can work in harmony with custom-coded solutions to deliver advanced, scalable functionality:
Low-Code Magic (Power Platform)
Power Apps: Provides a no-code interface where users interact with DobbAI, asking it to retrieve data, complete tasks, or run reports with just a few clicks.
Pro-Code Brilliance (SPFx and APIs)
SPFx Customizations: Extend SharePoint’s capabilities by embedding DobbAI into the site using modern web parts. These include pro-code components like advanced data handling and a rich user interface for AI-driven analytics.
Azure Active Directory V2 Authentication.
Low-Code Power
Copilot Studio: DobbAI’s foundation starts in Copilot Studio, where its capabilities are defined and fine-tuned to serve users effectively.
Power Platform:
Power Apps hosts DobbAI, making it easily accessible to all users in our organization.
Pro-Code Customizations
React PCF Component: The iframe provided by DobbAI’s custom website channel is integrated using a React PCF component, enabling advanced UI customization and smooth user experiences within Power Apps.
SharePoint Integration: By connecting to SharePoint via Microsoft Graph API and SharePoint REST API, DobbAI taps into the heart of our intranet for data retrieval.
Since many of the business owners of Diagon Alley spend most of the days on their feet in the store, they require a working solution that is easy to use and accessible wherever they might be. Because of this, we have decided to give them different possible solution within the Power Platform. Power Page for customer interaction, Power Automate to help reducing time demanding tasks and a data driven Power App that fit perfectly on a small Teams application on a store-owners cellphone.
Automating the office work
The daily-to-day operations of any business is to make sure there always is enough of their products, and making sure they always are stocked. To help the owners keep the supplies under control, we developed an Power Automate flow that would help them generate an Supplier Agreement contract in SharePoint.
The initial thought was to create a content type on a document library that would inherit metadata properties from the SharePoint columns and automatically fill inn the SharePoint property fields and thereby create a valid contract.
Unfortunately, this require that we edit the template locally, but we are working on computers with a safety policy that doesn’t allow us to connect to this fields when they are in another tenant. Still, we found a solution by downloading an empty document from the library, populate the fields quick parts connected to the document properties and re-upload this to the library. This allowed us to generate the agreement anyway, by using a Power Automate flow that populated the documents quick part fields.
When a pro-code developer creates magical code, we low-code developers need to assist where we can to speed up the work. We have created a good solution based on Azure AI Services, where we send data from Dataverse via Power Automate to a Python Flask app that translates everything we send into a message delivered to our Howler. Given the time we had, this would not have been possible.
The physical Howler integration is approaching, and the Arduino is working hard. It’s put together by paper, tape and glue. And much love! Here we have an example of what we have in store:
An example of translation and delivery of message follows: “I don’t like you anymore. You have proven to be untrustful!” -> “Dear Sirius,
Oh, how the tides have turned! I find myself in a whirlwind of emotions, and I must confess, I don’t like you anymore. Your betrayal stings like a Cruciatus curse, and it’s clear you’ve proven to be utterly untrustworthy!
How could you, of all people, turn your back on family? It’s as if you’ve taken a page from the Marauder’s playbook, but this time, it’s not a prank—it’s a dagger to the heart!
Desperation claws at me, and I can’t help but feel the madness creeping in, like a Dementor’s chill. You’ve left me no choice but to sever the ties that bind us.
Goodbye, Sirius. May you find your way back to the light, though I fear it may be too late for you.
This blog post will walk you through how to leverage both low-code tools and traditional coding to build a fun and effective user registration system. With the help of Microsoft Power Pages and Power Automate, we created a low-code solution that seamlessly integrates pro-code customization for a personalized experience.
Let’s break down in short lines the user flow: 1. The participant will fill out a registration form on the Power Pages site. 2. Their answers to a series of questions will be stored. 3. The user will be automatically sorted into one of the Houses based on their responses.
In Power Automate, we created a flow that is triggered by form submission from Power Pages (when user presses Join button). Then we added pro-code customization that packs the data and sends it to Dataverse.
To easly use the link mobility api we create a custom connector to integrate towards Link Moblity. To make sure we can keep track and controll the sms integration flow we use a message based integration with azure service bus and azure functions to handle the outgoing sms
As the figure over show:
With any kind of power automate flow, a custom trigger is avaiable to use to send sms towards link mobility
The Custom trigger is configured towards an azure function http trigger that taket the sms information and send it to a service bus
Another service bus trigger function picks up the message when and sendt it towards the link mobility api
A Service bus based sms integration provides a more robust system, as we are able in the backend to have control of the sms flow,
PowerPotters of Cepheo: Brewing Badge-Winning Elixirs with Pro-Code Potions and Beyond!”
Greetings, magical tech community! 🧙♂️✨ Team PowerPotters of Cepheo is thrilled to unveil our progress in automating elixir production for the ACDC 2025 hackathon. Our solution blends the powers of pro-code Python and low-code Power Platform to craft a system that’s both functional and badge-worthy. Today, we’ll reveal how we’re targeting the Pro-Code Potions category as well as these coveted badges:
Right Now
ACDC Craftsman
Thieving Bastards
Power User Love
Let’s dive into the details of how our Python-powered magic aligns with these badge aspirations!
1. Claiming the “Right Now” Badge: Python – The Heart of Our Brew
The “Right Now” badge rewards smart, clean, and efficient code that elevates solutions beyond the realm of low-code alone. For us, Python isn’t just an enhancement—it’s the engine of our magical elixir automation.
Python Scripts: Unlocking Advanced Functionality
Our Python scripts (sensor_script.py, voice_script.py, integration_script.py) are designed for tasks that Power Platform cannot handle natively:
Direct Hardware Interaction: Using RPi.GPIO, our sensor_script.py captures real-time data from the potion cauldron’s liquid level sensor. Power Platform simply cannot replicate this hardware integration.
AI-Powered Voice Recognition: Our voice_script.py leverages the OpenAI Whisper API for advanced speech-to-text processing, turning verbal commands into actionable automation triggers.
Intelligent Orchestration: The integration_script.py ties everything together—sensor readings, voice commands, and workflows via Power Automate.
With these examples, we demonstrate how Python serves as the lifeblood of our system, embodying the essence of “Right Now.”
2. The “ACDC Craftsman” Badge: Best Practices in Code
The “ACDC Craftsman” badge celebrates development and deployment excellence. Our commitment to best practices is reflected in every line of Python code we write.
Highlights of Craftsmanship:
Modular Structure: Each script (sensor_script.py, voice_script.py, etc.) has a single responsibility, ensuring clarity and maintainability.
Error Handling and Logging: Robust try...except blocks and detailed logging ensure stability and traceability
Mocking for Testability: The Mock folder includes a GPIO.py mock module, allowing us to test sensor logic without a physical Raspberry Pi. This approach accelerates development while maintaining code quality.
By embracing modularity, testability, and robust error handling, we ensure our code stands as a shining example of “ACDC Craftsman” principles.
3. The “Thieving Bastards” Badge: Leveraging External Tools and APIs
The “Thieving Bastards” badge rewards the clever use of third-party tools to amplify solutions. Here’s how we “borrowed” brilliance:
Open-Source Libraries:
RPi.GPIO and requests for hardware and API interaction.
sounddevice and scipy for audio recording and processing.
python-dotenv for secure environment variable management.
OpenAI Whisper API: This external AI service powers our voice recognition functionality, enabling seamless integration of advanced speech-to-text capabilities without reinventing the wheel.
We’ve strategically combined these tools to accelerate development and expand functionality, earning our place as “Thieving Bastards” in the best sense!
4. The “Power User Love” Badge: Pro-Code and Low-Code Unite
The “Power User Love” badge highlights the magic that happens when pro-code customization enhances low-code platforms. Our project is a perfect example:
Power Platform for Low-Code Power: Power Automate orchestrates workflows, while Power BI visualizes potion progress.
Python for Pro-Code Power: Python bridges the physical and digital realms, enabling sensor integration and AI-driven voice commands.
Together, these platforms create a seamless, intelligent, and user-friendly potion production system.
Conclusion: Badge-Winning Elixir Automation!
With our meticulously crafted Python pro-code and Power Platform low-code synergy, we’re confident our solution is a contender for:
Right Now
ACDC Craftsman
Thieving Bastards
Power User Love
PowerPotters of Cepheo are proud to combine technical excellence with magical creativity. We look forward to seeing the results and continuing to share our journey. Stay tuned, fellow wizards!
