Stairway To Heaven🤹🔗

Posted on by

Orchestrating Multiple Microsoft Cloud APIs for Intelligent Code Assistance

We wanted to build an intelligent code assistant that could answer questions about our codebase, help developers understand complex implementations, and enable business users to report bugs seamlessly. But to make this work, we needed to combine multiple Microsoft cloud services, each serving a different purpose, each with its own API, each requiring careful orchestration.

We needed to climb the stairway to cloud integration heaven.

The Solution: Orchestrating Six Microsoft Cloud APIs

We built a solution that seamlessly combines Azure AI Search, Azure OpenAI, Azure DevOps, Azure Key Vault, Azure Storage, and Azure Functions, all working together to create an intelligent, context-aware assistant that understands code and helps users effectively.

Here’s how we orchestrated these APIs to create something greater than the sum of its parts.

The Architecture: Six APIs, One Unified Experience

Our solution integrates multiple Microsoft cloud APIs:

Each API plays a crucial role, and together they create a seamless intelligent assistant experience.

Visual Architecture Diagram

Here’s a detailed visual representation of how all six APIs work together:

Here's a detailed visual representation of how all six APIs work together:

Step 1: Azure DevOps API – Code Repository Access

We start by accessing code from Azure DevOps repositories. The DevOps API lets us clone repositories, read files, and track changes, all programmatically.

Cloning Repositories with Azure DevOps API

import git
from azure.identity.aio import DefaultAzureCredential
from azure.keyvault.secrets.aio import SecretClient

class GitRepoStrategy:
    def __init__(self, organization, project, repository, branch="main", pat=None, key_vault_url=None):
        self.organization = organization
        self.project = project
        self.repository = repository
        self.branch = branch
        self.pat = pat
        self.key_vault_url = key_vault_url

    async def _get_pat(self):
        """Retrieve Personal Access Token from Key Vault or environment."""
        if self.pat:
            return self.pat

        if self.key_vault_url:
            credential = DefaultAzureCredential()
            async with SecretClient(vault_url=self.key_vault_url, credential=credential) as client:
                secret = await client.get_secret("azure-devops-pat")
                return secret.value

        return os.getenv("AZURE_DEVOPS_PAT")

    async def _clone_repository(self):
        """Clone repository using GitPython with PAT authentication."""
        pat = await self._get_pat()
        repo_url = f"https://{pat}@dev.azure.com/{self.organization}/{self.project}/_git/{self.repository}"

        temp_dir = tempfile.mkdtemp(prefix="git_repo_")
        logger.info(f"Cloning repository {self.organization}/{self.project}/{self.repository} to {temp_dir}")

        git.Repo.clone_from(repo_url, temp_dir, branch=self.branch)
        return temp_dir

This code:

  • Uses Azure Key Vault API to securely retrieve the Personal Access Token
  • Uses Azure DevOps Git API to clone repositories
  • Handles authentication seamlessly across services

Creating Work Items with Azure DevOps API

import aiohttp
import base64
from azure.identity.aio import DefaultAzureCredential

async def create_devops_workitem(title, description, work_item_type="Bug"):
    """Create a work item in Azure DevOps using REST API."""

    # Get PAT from Key Vault (Azure Key Vault API)
    credential = DefaultAzureCredential()
    key_vault_url = os.getenv("AZURE_KEYVAULT_URL")

    async with SecretClient(vault_url=key_vault_url, credential=credential) as client:
        secret = await client.get_secret("azure-devops-pat")
        pat = secret.value

    # Prepare Azure DevOps API request
    organization = os.getenv("AZURE_DEVOPS_ORG")
    project = os.getenv("AZURE_DEVOPS_PROJECT")

    auth_header = base64.b64encode(f":{pat}".encode()).decode()
    url = f"https://dev.azure.com/{organization}/{project}/_apis/wit/workitems/${work_item_type}?api-version=7.1"

    headers = {
        "Authorization": f"Basic {auth_header}",
        "Content-Type": "application/json-patch+json",
    }

    patch_document = [
        {
            "op": "add",
            "path": "/fields/System.Title",
            "value": title
        },
        {
            "op": "add",
            "path": "/fields/System.Description",
            "value": description
        }
    ]

    async with aiohttp.ClientSession() as session:
        async with session.patch(url, headers=headers, json=patch_document) as response:
            if response.status in [200, 201]:
                work_item_data = await response.json()
                work_item_id = str(work_item_data.get("id", ""))
                work_item_url = work_item_data.get("_links", {}).get("html", {}).get("href", "")
                return {"workItemId": work_item_id, "workItemUrl": work_item_url}

This demonstrates:

  • Azure Key Vault API for secure secret retrieval
  • Azure DevOps REST API for work item creation
  • Seamless integration between the two services

Step 2: Azure Storage API – Document Storage and Retrieval

Once we have code files, we need to store them and make them accessible. Azure Storage provides blob storage for documents and code files.

Storing Documents with Azure Storage API

from azure.storage.blob.aio import BlobServiceClient
from azure.identity.aio import DefaultAzureCredential

class BlobManager:
    def __init__(self, account_name, container_name, credential=None):
        self.account_name = account_name
        self.container_name = container_name
        self.credential = credential or DefaultAzureCredential()

        account_url = f"https://{account_name}.blob.core.windows.net"
        self.blob_service_client = BlobServiceClient(
            account_url=account_url,
            credential=self.credential
        )

    async def upload_file(self, file_path, blob_name):
        """Upload a file to Azure Blob Storage."""
        async with self.blob_service_client:
            blob_client = self.blob_service_client.get_blob_client(
                container=self.container_name,
                blob=blob_name
            )

            with open(file_path, "rb") as data:
                await blob_client.upload_blob(data, overwrite=True)

            return blob_client.url

    async def list_blobs(self, prefix=""):
        """List all blobs in the container."""
        async with self.blob_service_client:
            container_client = self.blob_service_client.get_container_client(
                self.container_name
            )

            blobs = []
            async for blob in container_client.list_blobs(name_starts_with=prefix):
                blobs.append({
                    "name": blob.name,
                    "size": blob.size,
                    "last_modified": blob.last_modified
                })

            return blobs

This uses:

  • Azure Storage Blob API for file storage
  • Azure Identity API for authentication
  • Managed identity for seamless access

Step 4: Azure AI Search API – Intelligent Search and Indexing

Azure AI Search provides the search infrastructure that makes our code searchable and retrievable.

Indexing Code with Azure AI Search API

from azure.search.documents.aio import SearchClient
from azure.search.documents.indexes.aio import SearchIndexClient
from azure.search.documents.models import IndexingResult
from azure.identity.aio import DefaultAzureCredential

class SearchManager:
    def __init__(self, search_endpoint, index_name, credential=None):
        self.search_endpoint = search_endpoint
        self.index_name = index_name
        self.credential = credential or DefaultAzureCredential()

        self.search_client = SearchClient(
            endpoint=search_endpoint,
            index_name=index_name,
            credential=self.credential
        )

    async def index_document(self, document):
        """Index a document in Azure AI Search."""
        async with self.search_client:
            result = await self.search_client.upload_documents(documents=[document])
            return result[0]

    async def search(self, query, top=5):
        """Search the index."""
        async with self.search_client:
            results = await self.search_client.search(
                search_text=query,
                top=top,
                include_total_count=True
            )

            documents = []
            async for result in results:
                documents.append({
                    "content": result.get("content"),
                    "filename": result.get("filename"),
                    "score": result.get("@search.score")
                })

            return documents

    async def create_index(self):
        """Create the search index if it doesn't exist."""
        from azure.search.documents.indexes.models import (
            SearchIndex,
            SimpleField,
            SearchFieldDataType,
            SearchableField
        )

        index_client = SearchIndexClient(
            endpoint=self.search_endpoint,
            credential=self.credential
        )

        index = SearchIndex(
            name=self.index_name,
            fields=[
                SimpleField(name="id", type=SearchFieldDataType.String, key=True),
                SearchableField(name="content", type=SearchFieldDataType.String),
                SimpleField(name="filename", type=SearchFieldDataType.String),
                SimpleField(name="category", type=SearchFieldDataType.String),
            ]
        )

        async with index_client:
            await index_client.create_or_update_index(index)

This demonstrates:

  • Azure AI Search API for indexing and searching
  • Azure Identity API for authentication
  • Vector search capabilities for semantic understanding

Step 5: Azure OpenAI API – Language Understanding and Generation

Azure OpenAI provides the language model that understands questions and generates intelligent responses.

Generating Answers with Azure OpenAI API

from openai import AsyncOpenAI
from azure.identity.aio import get_bearer_token_provider, DefaultAzureCredential

class OpenAIClient:
    def __init__(self, endpoint, deployment_name, api_version="2024-02-15-preview"):
        credential = DefaultAzureCredential()
        token_provider = get_bearer_token_provider(
            credential,
            "https://cognitiveservices.azure.com/.default"
        )

        self.client = AsyncOpenAI(
            api_key="placeholder",  # Not used with token provider
            azure_endpoint=endpoint,
            azure_ad_token_provider=token_provider,
            api_version=api_version
        )
        self.deployment_name = deployment_name

    async def generate_embeddings(self, text):
        """Generate embeddings for text using Azure OpenAI."""
        response = await self.client.embeddings.create(
            model=self.deployment_name,
            input=text
        )
        return response.data[0].embedding

    async def chat_completion(self, messages, temperature=0.3):
        """Generate chat completion using Azure OpenAI."""
        response = await self.client.chat.completions.create(
            model=self.deployment_name,
            messages=messages,
            temperature=temperature
        )
        return response.choices[0].message.content

This uses:

  • Azure OpenAI API for language understanding
  • Azure Identity API for token-based authentication
  • Embeddings for semantic search
  • Chat completions for natural language responses

Step 6: Azure Key Vault API – Secure Secret Management

Azure Key Vault securely stores secrets like API keys and connection strings, accessed by other services.

Retrieving Secrets with Azure Key Vault API

from azure.keyvault.secrets.aio import SecretClient
from azure.identity.aio import DefaultAzureCredential

class SecretManager:
    def __init__(self, vault_url):
        self.vault_url = vault_url
        self.credential = DefaultAzureCredential()

    async def get_secret(self, secret_name):
        """Retrieve a secret from Azure Key Vault."""
        async with SecretClient(vault_url=self.vault_url, credential=self.credential) as client:
            secret = await client.get_secret(secret_name)
            return secret.value

    async def set_secret(self, secret_name, secret_value):
        """Store a secret in Azure Key Vault."""
        async with SecretClient(vault_url=self.vault_url, credential=self.credential) as client:
            await client.set_secret(secret_name, secret_value)

This demonstrates:

  • Azure Key Vault API for secret management
  • Azure Identity API for authentication
  • Secure access to sensitive credentials

The Complete Integration: Six APIs Working Together

Now let’s see how all six APIs work together in a complete flow:

Complete Code Indexing Flow via Azure Functions

# This runs in Azure Functions - triggered by HTTP request
async def index_code_repository():
    """Complete flow combining all six APIs, orchestrated by Azure Functions."""

    # Step 1: Get secrets from Azure Key Vault API
    credential = DefaultAzureCredential()
    key_vault_url = os.getenv("AZURE_KEYVAULT_URL")

    async with SecretClient(vault_url=key_vault_url, credential=credential) as kv_client:
        devops_pat = await kv_client.get_secret("azure-devops-pat")
        storage_key = await kv_client.get_secret("storage-account-key")

    # Step 2: Clone repository using Azure DevOps API
    git_strategy = GitRepoStrategy(
        organization=os.getenv("AZURE_DEVOPS_ORG"),
        project=os.getenv("AZURE_DEVOPS_PROJECT"),
        repository=os.getenv("AZURE_DEVOPS_REPO"),
        pat=devops_pat.value
    )
    repo_path = await git_strategy._clone_repository()

    # Step 3: Process files and upload to Azure Storage API
    blob_manager = BlobManager(
        account_name=os.getenv("AZURE_STORAGE_ACCOUNT"),
        container_name=os.getenv("AZURE_STORAGE_CONTAINER"),
        credential=credential
    )

    files_processed = 0
    async for file in git_strategy.list():
        # Parse file content
        sections = await parse_file(file, file_processors, category="code")

        # Upload to storage
        blob_name = f"code/{file.filename}"
        await blob_manager.upload_file(file.path, blob_name)

        # Step 4: Generate embeddings using Azure OpenAI API
        openai_client = OpenAIClient(
            endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
            deployment_name=os.getenv("AZURE_OPENAI_EMB_DEPLOYMENT")
        )

        embeddings = []
        for section in sections:
            embedding = await openai_client.generate_embeddings(section.content)
            embeddings.append(embedding)

        # Step 5: Index documents in Azure AI Search API
        search_manager = SearchManager(
            search_endpoint=os.getenv("AZURE_SEARCH_ENDPOINT"),
            index_name=os.getenv("AZURE_SEARCH_INDEX"),
            credential=credential
        )

        for i, section in enumerate(sections):
            document = {
                "id": f"{file.filename}_{i}",
                "content": section.content,
                "filename": file.filename,
                "category": "code",
                "embedding": embeddings[i]
            }
            await search_manager.index_document(document)

        files_processed += 1

    return {"files_processed": files_processed}

This complete flow demonstrates:

  1. Azure Functions API – Serverless orchestration and execution
  2. Azure Key Vault API – Retrieving secrets securely
  3. Azure DevOps API – Cloning and accessing code repositories
  4. Azure Storage API – Storing documents and files
  5. Azure OpenAI API – Generating embeddings and understanding code
  6. Azure AI Search API – Indexing and making code searchable

The Chat Flow: APIs Working in Harmony via Functions

When a user asks a question, Azure Functions orchestrates all APIs to work together:

async def answer_question(question: str):
    """Answer a question using all integrated APIs."""

    # Step 1: Search using Azure AI Search API
    search_manager = SearchManager(
        search_endpoint=os.getenv("AZURE_SEARCH_ENDPOINT"),
        index_name=os.getenv("AZURE_SEARCH_INDEX"),
        credential=DefaultAzureCredential()
    )

    # Generate query embedding using Azure OpenAI API
    openai_client = OpenAIClient(
        endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
        deployment_name=os.getenv("AZURE_OPENAI_EMB_DEPLOYMENT")
    )

    query_embedding = await openai_client.generate_embeddings(question)

    # Search with vector similarity
    search_results = await search_manager.vector_search(
        vector=query_embedding,
        top=5
    )

    # Step 2: Generate answer using Azure OpenAI API
    chat_client = OpenAIClient(
        endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
        deployment_name=os.getenv("AZURE_OPENAI_CHAT_DEPLOYMENT")
    )

    messages = [
        {
            "role": "system",
            "content": "You are a helpful assistant that answers questions about code."
        },
        {
            "role": "user",
            "content": f"Question: {question}\n\nContext:\n{format_search_results(search_results)}"
        }
    ]

    answer = await chat_client.chat_completion(messages)

    # Step 3: If user wants to create a bug, use Azure DevOps API
    if "create bug" in question.lower() or "report issue" in question.lower():
        # Get PAT from Azure Key Vault API
        async with SecretClient(
            vault_url=os.getenv("AZURE_KEYVAULT_URL"),
            credential=DefaultAzureCredential()
        ) as kv_client:
            pat = await kv_client.get_secret("azure-devops-pat")

        # Create work item using Azure DevOps API
        work_item = await create_devops_workitem(
            title=extract_bug_title(question),
            description=answer,
            work_item_type="Bug"
        )

        return {
            "answer": answer,
            "work_item_id": work_item["workItemId"],
            "work_item_url": work_item["workItemUrl"]
        }

    return {"answer": answer, "sources": search_results}

This demonstrates the complete orchestration:

  • Azure Functions API orchestrates the entire workflow serverlessly
  • Azure AI Search API finds relevant code
  • Azure OpenAI API understands the question and generates embeddings
  • Azure OpenAI API generates intelligent answers
  • Azure Key Vault API provides secure access to secrets
  • Azure DevOps API creates work items when needed

The Benefits: More Than the Sum of Parts

Individual API Capabilities

Each API alone is powerful:

  • Azure Functions API: Serverless compute and orchestration
  • Azure DevOps API: Repository access and work item management
  • Azure Storage API: Scalable file storage
  • Azure AI Search API: Fast, intelligent search
  • Azure OpenAI API: Language understanding and generation
  • Azure Key Vault API: Secure secret management

Combined Power

Together, they create something greater:

  • Intelligent Code Assistant: Understands code context and answers questions
  • Seamless Integration: All APIs work together with shared authentication
  • Secure by Default: Key Vault manages all secrets
  • Scalable Architecture: Each service scales independently
  • Unified Experience: Users see one intelligent assistant, not five separate APIs

The Technical Architecture

Our solution architecture:

┌─────────────────┐
│   User Query    │
└────────┬────────┘
         │
         ▼
┌─────────────────────────────────────┐
│      Azure Identity API             │
│   (Unified Authentication)          │
└────────┬────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────┐
│      Azure Functions API            │
│   (Serverless Orchestration)         │
└────────┬────────────────────────────┘
         │
         ├──────────────────────────────────┐
         │                                  │
         ▼                                  ▼
┌──────────────────┐            ┌──────────────────┐
│ Azure Key Vault  │            │  Azure OpenAI    │
│      API         │            │      API         │
│  (Secrets)       │            │  (Embeddings)    │
└────────┬─────────┘            └────────┬─────────┘
         │                                │
         │                                ▼
         │                    ┌──────────────────┐
         │                    │  Azure AI Search │
         │                    │      API         │
         │                    │   (Indexing)     │
         │                    └────────┬─────────┘
         │                             │
         │                             ▼
         │                    ┌──────────────────┐
         │                    │  Azure Storage   │
         │                    │      API         │
         │                    │   (Documents)    │
         │                    └────────┬─────────┘
         │                             │
         └─────────────────────────────┘
                     │
                     ▼
         ┌──────────────────────┐
         │  Azure DevOps API    │
         │  (Work Items)        │
         └──────────────────────┘

Authentication: The Unifying Thread

All APIs use Azure Identity API for unified authentication. Azure Functions leverages managed identity to access all other services:

from azure.identity.aio import DefaultAzureCredential

# In Azure Functions, managed identity is automatically used
# One credential works for all services
credential = DefaultAzureCredential()

# Use with Key Vault
kv_client = SecretClient(vault_url=key_vault_url, credential=credential)

# Use with Storage
blob_client = BlobServiceClient(account_url=storage_url, credential=credential)

# Use with Search
search_client = SearchClient(endpoint=search_endpoint, credential=credential)

# Use with OpenAI (via token provider)
token_provider = get_bearer_token_provider(
    credential,
    "https://cognitiveservices.azure.com/.default"
)
openai_client = AsyncOpenAI(azure_ad_token_provider=token_provider)

# Use with DevOps (via PAT from Key Vault)
pat = await kv_client.get_secret("azure-devops-pat")

One credential. Six APIs. Seamless integration through Azure Functions.

Real-World Example: Bug Reporting Flow

Let’s trace a complete bug reporting flow orchestrated by Azure Functions:

  1. User asks: “The checkout button isn’t working. Why?”
  2. Azure Functions API receives the request and orchestrates the workflow:
  • Triggers the answer generation process
  • Manages the entire flow serverlessly
  1. Azure AI Search API finds relevant code:
  • Searches index for “checkout button”
  • Returns code snippets from checkout components
  1. Azure OpenAI API analyzes the code:
  • Generates embeddings for the question
  • Finds semantically similar code
  • Understands the context
  1. Azure OpenAI API generates explanation:
  • Analyzes the code snippets
  • Explains why the issue might be occurring
  • Provides technical details
  1. Azure Key Vault API provides secure access:
  • Retrieves DevOps PAT securely
  • No secrets in code or environment variables
  1. Azure DevOps API creates work item:
  • Creates bug with title and description
  • Links to relevant code files
  • Returns work item URL

All six APIs working together seamlessly, orchestrated by Azure Functions.

The Code Repository

Our implementation includes:

  • app/functions/code_sync/function_app.py – Azure Functions orchestrating code sync
  • app/functions/code_sync/host.json – Function App configuration
  • app/backend/app.py – Main application calling Functions and orchestrating APIs
  • app/backend/prepdocslib/gitrepostrategy.py – Azure DevOps API integration
  • app/backend/prepdocslib/searchmanager.py – Azure AI Search API integration
  • app/backend/prepdocslib/blobmanager.py – Azure Storage API integration
  • app/backend/approaches/chatreadretrieveread.py – Azure OpenAI API integration
  • scripts/sync_code_repo.py – Complete orchestration script

All APIs integrated. All working together through Azure Functions orchestration.

The Business Impact

Before API integration:

  • Manual processes: Copy-paste code, manual searches
  • Disconnected tools: Separate tools for each task
  • No intelligence: Can’t understand code context
  • Slow workflows: Multiple steps, multiple tools

After API integration:

  • Automated workflows: One query triggers multiple APIs
  • Unified experience: One intelligent assistant
  • Context-aware: Understands code and relationships
  • Fast responses: APIs work in parallel

We went from disconnected tools to an intelligent, integrated solution.

What’s Next

API integration opens new possibilities:

  • Multi-repository support: Index code from multiple DevOps projects
  • Real-time updates: Sync code changes automatically
  • Advanced analytics: Understand code patterns and trends
  • Cross-service insights: Combine data from multiple sources
  • Intelligent automation: APIs trigger each other automatically

But the foundation is set: we have seamless integration across six Microsoft cloud APIs. Everything else builds on this.

This API orchestration demonstrates the power of combining Microsoft cloud services. By integrating Azure Functions, Azure DevOps, Azure Storage, Azure AI Search, Azure OpenAI, and Azure Key Vault, all unified through Azure Identity, we’ve created an intelligent code assistant that’s more than the sum of its parts. Six APIs. One unified experience. Orchestrated serverlessly. The stairway to cloud integration heaven.