Tag Archives: Stairway To Heaven

Crashing the Azure AI Studio and Copilot

Posted on by

Badges to claim:

  • Nasty Hacker – for retrieving data from 3rd party services and syncing it to the blob storage to connect to Azure AI Studio
  • Data miner – for retrieving the case information from dataverse and calculating the average score using AI.
  • Embedding numbnut – for embedded copilot in Model-driven app
  • Stairway to heaven – for using Azure AI Studio, Copilot, Blob storage, and in previous articles also Azure Function, WebApp

Our solution includes the latest features of the PowerPlatform and Azure connection the Low and Pro code approaches together, to allows you to boost the performance of resolving the cases by using some insights from the Copilot. 

  • The business use case is about complicated cases when we need external consultancy and assistant, so task is to find the suitable Marios-consultants according to the customer request, by searching the professionals on the Indeed, comparing their background and experience and finding the best matches. (to find the suitable Marios according to the request.) 
  • The business use case is about simplify the KYC(Know Your Customer) process by using unified workspace for all operations.  From Indeed we can understand the company background,do semantic analysis of the comments to have insights on how technicians can approach customer (princess) (simplify the KYC(Know Your Customer) process by using unified workspace for all operations.) 
  • The business use case is about analyzing current princess (customer) and her needs based on the Indeed. The data that we were able to retrieve contains open job postings, company in for, recent news and personal profile info. Based on this information, we can suggest more other services and provide information to sales and marketing departments. (analyze current princess and her needs based on the Indeed profile and suggest more other services.) 

We deployed GPT-4 model to our personal instance using AI Studio, then fine-tuned it with company’s internal data and data from open sources like Indeed, Glassdor or proff.no. 

Than using Retrieval Augmented Generation (RAG) technic to inject generative responses from big LLM into answer of Copilot. 

With RAG, the external data used to augment your prompts can come from multiple data sources, such as a document repositories, databases, or APIs. The first step is to convert your documents and any user queries into a compatible format to perform relevancy search. To make the formats compatible, a document collection, or knowledge library, and user-submitted queries are converted to numerical representations using embedding language models. Embedding is the process by which text is given numerical representation in a vector space.  

AI Studio has seamless integrated feature use Azure Blob Indexer to implement search functionality search for AI. It bring the possibility to simplify the access to the Datalake from LLM side.   

By implementing multiple connectors to the third-party services and data sources, together with Dynamic chaining feature of the Copilot it gives cleaner user experience for the user using only one tool for analyses. 

How it works:

Loving the ease and use of Web API Connectors in Power Automate

Posted on by

In the following flow we have integrated three different connectors with the purpose of logging and giving an overview of DLP for all the ChatGPT related flows. We’ve involved user information through Azure Entra ID service, posting logs in Teams admin channel and also fetching the stored Azure Key Vault secret for security reasons which are then encrypted and hidden information based on the action settings of the HTTP action and Azure Key Vault action.

KoopaKonstruct Manager’s Stairway to Heaven

Posted on by

The Hololense device is not just a headset; it’s a portal to a world where the boundaries between the virtual and the real blur. To achieve this seamless integration, we leverage the prowess of four distinct Microsoft Cloud APIs, each playing a crucial role in orchestrating our KoopaKonstruct Manager adventure.

Power Automate: Automating the Journey

Power Automate acts as the orchestrator of our adventure, seamlessly connecting the various components of our solution. With Power Automate, we automate workflows that trigger updates in FinOps, generate purchase and sales orders, and send confirmation emails. Additionally, the flow sends inventory status back to the Hololenses for the user to be updated on inventory quantity on sight. This cloud-based automation engine ensures that our journey unfolds with precision and efficiency, leaving no room for manual intervention.

FinOps: Navigating the Financial Landscape

FinOps, a cloud financial management framework, guides us through the financial landscape of our project. By integrating FinOps, we ensure that the procurement of materials aligns with budgetary constraints. The cloud-based financial intelligence provided by FinOps becomes the compass that guides our decisions, ensuring a smooth journey through the financial complexities of our adventure.

Unity: Crafting the Virtual Reality

Unity, a powerful cross-platform game engine, becomes the canvas for our Hololens-powered adventure. Through Unity, we bring the Super Mario-themed world to life, creating a virtual playground where users can interact with their surroundings. Unity’s versatility allows us to seamlessly integrate Hololens functionalities, providing an immersive and interactive experience that blurs the lines between the physical and digital realms.

Uniting the Microsoft Cloud APIs

The integration of Power Automate, Unity, and FinOps with Hololens exemplifies the synergy achieved by uniting Microsoft Cloud APIs. Together, they form a cohesive ecosystem where data seamlessly flows, processes are automated, and a Super Mario-themed world comes to life before our eyes.

This integration is not just about earning a badge; it’s about unlocking the potential of cloud technologies to create experiences that transcend the ordinary. Our journey is a testament to the endless possibilities that unfold when diverse Microsoft Cloud APIs unite in a symphony of innovation!

Piping our way through the Azure Cloud

Posted on by

To make our amazing service Tubi work, a lot of cloud is needed. We aim to make the plumber’s job easier by recommending the best layout for where the pipes should go, and for that, we need AI. We have trained a model in Custom Vision to recognize all components in a bathroom that need water. So, when the plumber uploads a floor plan to our Static Web App, the image is sent to our Azure Function App backend in C# Asp.net through our own API. But both the image and the equipment list must be stored somewhere. Therefore, we have also connected to Azure Blob Storage. Then last but not at all least. The people working in the back office have instant interactive reports available to help them with filing and billing through Power BI and alerting the using an automated flow (Badges: Feature Bombing)

Sometimes it works, and that’s plenty

Databases are good, but sometimes it’s easier to just dump everything in one place until you need it again. Yes, it might not be very scalable or very normalized. SQL became too heavy, and we already needed a Blob storage to store the images, so we also dump the order data in the same blob storage as JSON files. It’s old fashioned way of serverstorage, and a bit dirty, but it works! (Badges: Nasty hacker, Retro badge)

Power the backoffice

As the final list of components are decided, they still have to be approved from the accounting team in the office. To make sure they have all the information they require, we have developed a Power BI dashboard to crawl through our registered data and make sure the orders are handled properly (Badges: Crawler, Dash it Out, Dataminer). And to make sure the orders are handled easy and fast, the dashboard is embedded into teams and an alert is automated by using a logic app to make sure the workers can receive and cooperate in realtime (Badges: Embedding Numbnuts, Go with the flow, Pug N’ Play, Power user love, Right Now, Stairway to heaven).

Climbing the Stairways to Heaven!

Posted on by

Since we build on Power Platform, Power Apps and Power Automate provides connectors to various APIs that we use in our app logic and user interface.

Here are some of the ways we are using APIs in our solution:

Power Automate and Outlook 365 API

We use the Outlook 365 API to access one user’s inbox and wait for an incoming email. This email is from the Game Player – and determines if the player progresses in the game or not. This is a bonus nested API call as the Power App triggers the use of the Outlook 365 API through Power Automate!

Copilot Studio

We use the Copilot Studio API to interact with a custom prompt that can act as an advisor throughout the game. At any point in time you can activate the chat and ask any questions you want, to get clues and hints on how to progress in the game. (The butler even knows a lot of secrets about the little town!)

Dataverse

We use the Dataverse API to store and retrieve data for our app/game in Dataverse Tables, such as game progress, Rooms/Locations, Excercises/Questions, Assets (images etc). This makes the game/app much more dynamic as we do not store the information within the app itself.

Star world to the cloud

Posted on by

Choo choo! All aboard the star train to the microsoft cloud! The sole action of finding and identifing bananas is not enough. We need to alert people of the danger that has been detected in traffic before they can cause more harm.

Introducing the Banana Notifier Bot (BnB):

The BnB helps our users by sending teams notifications when bananas have been detected.It uses our notification API (which gets data from our banana-scanning app), Bing Maps API and Graph API with location data and timestamp.

Our Notification API also utilizes the SharePoint REST Api to fetch next of kin information from a SharePoint list. Lastly, we store all our banana findings in Dataverse using the Dataverse Web API. Utilizing a total of four (!) Microsoft APIs.

Captain’s log – data at your hook-tip

Posted on by

The captain log business value is that he has an app with all the tools he needs at the tip of his hook.

A new feature:

Behold the “Reveal your true face feature”

Revealing the true face

Some pirates find their interest in hiding away, and then it is handy to have an app to show the true face and spot if this is a know fellow shipmate or a rivalling pirate that deserves to be keel hauled.

The service uses a mobile or tablet camera to take a picture of the person in-question and then posts it to mutiny services for identification and face recognition in order to find who this person is.

Claiming the ACDC Craftsman since it is created using actual code with some best practices in react development: https://github.com/TheBastards-ACDC2023/Bastards-NativeApp

Maritime Mischief Manager (MMM) – A description

Posted on by

“MMM.. is the ultimate tool for pirate captains looking to maximize their raids. With our comprehensive system, you’ll be able to plan, organize, and execute the perfect plunder. No more manual logs, no more disorganized crew. Just a streamlined, efficient process for taking what you want from the high seas. Upgrade your pirate game today with Maritime Mischief Manager.”

Pirates are met with a friendly user interface where they can manage all aspects of pillaging.

Creating a Raid

Under the “New Raid” menu, pirates with the rank of Captain can take a look at possible raid prospects and get an overview of the best possible strategy and also an estimation of the loot potential in the selected location.

The results we are seeing above comes from a query against the ChatGPT API with the use of Power Automate.

Being satisified with the current target, the Captain selects the vessel for the raid, choose to create it and the raid enters planning mode.

Raid planning

From the raid list, piratescan view raids in the planning stage. Simple pirates can choose to join a raid if they want to, but Captains can also choose to invite his best men. Pirates receiving an invitation will have to accept/reject the invitation through their Teams account.

Once the pirate has received and accepted an invitation, they have to scan their provided RFID card when they board the ship. They will then enter the status of Checked in.

Once the RFID card is scanned at the ship, a HTTP request is sent to the raid check-in flow, the flow will update the crew status.


When the Captain is happy with his band of criminals, he chooses to start the raid. The app conveniently simulates the battle as well, together with some nifty battle music. This is of course to save the hassle of actually raiding a place.

Reporting

The pirate management team of course also have access to beautiful numbers of the entire operation

Stairway to Heaven for Orange Bandits

Posted on by

Leveraging of Microsoft API’s allows developers to quickly get started witch achieving meaningful results in the development process.

Here is a quick summary of API’s we used, that are delivered by Microsoft:

  • Dataverse REST API:

Used as backbone of our systems integrations, main data ingest and source point. It is used in PowerBI to expose data to report:

  • Microsoft Bot API

It is used in sending notifications to Teams App (more on that in A Pirate Call to Arms! | Arctic Cloud Developer Challenge Submissions (acdc.blog)!).

  • Machine Learning Services API

ML REST API is used in PowerBI report to show the information about potential pirate looting activity outcome, based on information about the potential looting target:

Microsoft Identity platform (Azure Active Directory Authentication)

call from .NET code
and as it’s seen in Fiddler

This vehicleApi is on top of microsoft sql server database which is providing vehicle data based on MachineLearnings recognition of a make of a car. In order for the api to work, it connects to Azure Keyvault – and in order to access it, a call to microsoft Azure Ad Authentication API is made.

🚀 Automagical Fairs and Shouts to Fellow Pirats using Service Slave Workers 📣

Posted on by

When plundering and sailing keeping track of pesky DM’s is no job for the captain or crew. This’ why we summoned the services of Power Automate and Power Apps to do our bidding: Enabling our ship to run smoothly and commands being forwarded at the correct time to the correct people.

It all starts with the Consumer of Data “Power Automate” who collects data from all our different intelligence points. Working in the shadows, this ghost can be attached to any workflow. Currently, since it thrives on human flesh for its operations, we started by attached to our crew log. Whenever Power Automate smells the fear of a new deck hand, laddie or lassie onboard it goes to work!

On a new pirate record Power Automate does two things:

  1. Creates an Adaptive Card in teams and posted it to the ships group chat
  2. Sends a welcome e-mail to the fresh pirate, explaining how they need to obey or walk the plank (all pretty standard pirate stuff).
Current Dataverse Tables the flow can write to and read from.
Power automate flows that creates and updates records, when records for pirates are updated we trigger a Teams Integration.

Making the Captain’s work Screen Agnostic

We further improve the Captain’s life with Power Apps connected to the Power Flows. The Power App sits on the users phone, implemented via Teams or as a standalone app, and enables the Captain to administrate the crew roster, so any stowaway can be added to the crew and receive the required e-mail package with plank rules and NDA’s. This mobile approach lets the captain be in full control even when venturing away from his cabin and the desktop computer therein.

Screen snippet showing Power Automate triggering Teams Adaptive Card on new shipmate record.

The convenience of having the Power Apps in teams means the captain can manipulate his Ship, The Raids and roster from any surface, be it mobile, desktop og even tablet!

Screen snippet showing the Power App integrated into a Teams. Notice the responsive nature of teams adapting to any viewport.

Using Power Apps means we can also simulate raids adding new test data on the fly, without it needing to come from our Power Automate flow.

User generated Raids in the raid table, created directly within Power App.

In Summary, for Sleepy Pirates:

  • The Power App enables our users to directly manipulate our data storage.
  • The Power Automate allow other sources and application to remotely access our data.