The Existential Risk🔐👮

Posted on by

Building AI with Conscience, Reasoning, and Self-Awareness

The Challenge: Can AI Think? Does It Have a Conscience?

When building AI systems, we face fundamental questions: Can AI think on its own? Does it have a conscience? Is it smarter than a 5th grader? These are practical concerns that determine whether AI systems are safe, trustworthy, and truly intelligent.

We needed to build an AI system that:

  • Thinks independently: Can reason through problems without just pattern matching
  • Has a conscience: Understands ethical boundaries and refuses harmful requests
  • Demonstrates intelligence: Can reason at different levels, adapting complexity to the problem
  • Shows self-awareness: Understands its own reasoning process and can explain it

We needed AI that doesn’t just answer—it thinks, evaluates, and reasons.

The Solution: Reasoning Models, Agentic Behavior, and Safety Evaluation

We built an AI system using Azure OpenAI reasoning models, agentic retrieval, and comprehensive safety evaluation. Our solution demonstrates AI that can think independently, evaluate ethical boundaries, and reason at multiple complexity levels—showing both intelligence and conscience.

Here’s how we implemented it.

The Architecture: Three Layers of Intelligence

Our AI system has three layers that work together:

User Query → Reasoning Layer → Agentic Planning → Safety Evaluation → Response
     ↓              ↓                  ↓                  ↓              ↓
  Question    Think First      Plan Search        Check Ethics    Safe Answer
              (Reasoning)      (Agentic)          (Conscience)    (Intelligent)

Each layer adds a different dimension of intelligence and safety.

The Technical Architecture

┌─────────────────────────────────────────────────────────────┐
│                    USER QUERY                                │
│  "What are the risks of AI?"                                │
└──────────────────────┬──────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────┐
│              REASONING LAYER                                 │
│  • GPT-5 / O3 / O1 Models                                   │
│  • Internal Thinking Process                                 │
│  • Reasoning Effort: minimal/low/medium/high                │
│  • Thought Process Tracking                                  │
└──────────────────────┬──────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────┐
│           AGENTIC PLANNING LAYER                             │
│  • Analyzes Conversation Context                             │
│  • Plans Search Strategy                                     │
│  • Generates Multiple Queries                                │
│  • Autonomous Decision Making                                │
└──────────────────────┬──────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────┐
│        FILTER & ACCESS CONTROL LAYER                         │
│  • Metadata Filtering (project/repo/customer)                │
│  • User Identity Validation                                  │
│  • Group Membership Checking                                 │
│  • Permission Enforcement                                    │
└──────────────────────┬──────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────┐
│           SAFETY EVALUATION LAYER                            │
│  • Ethical Boundary Checking                                 │
│  • Harmful Content Detection                                 │
│  • Safety Scoring                                            │
│  • Conscience Validation                                     │
└──────────────────────┬──────────────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────────────┐
│              RESPONSE GENERATION                             │
│  • Synthesizes Information                                   │
│  • Applies Ethical Filters                                   │
│  • Generates Safe, Intelligent Answer                        │
│  • Exposes Thought Process                                   │
│  • Respects Data Boundaries                                  │
└─────────────────────────────────────────────────────────────┘

Step 1: Reasoning Models – AI That Thinks Before Answering

We implemented reasoning models that spend time “thinking” before generating responses. Unlike traditional models that generate answers immediately, reasoning models process and understand requests first.

Implementation

We integrated Azure OpenAI reasoning models (GPT-5, O3, O1) that use a “thinking” process:

Configuration:

  • Model Selection: Deployed reasoning models (GPT-5, O3-mini, O1) that support internal reasoning
  • Reasoning Effort Levels: Configurable thinking depth (minimal, low, medium, high)
  • Thought Process Visibility: Users can see the AI’s reasoning process and token usage

How It Works:

The reasoning model receives a query and:

  1. Thinks internally: Processes the question, considers context, evaluates options
  2. Plans approach: Decides how to structure the answer
  3. Generates response: Produces the final answer based on reasoning

Key Implementation Details:

# Reasoning effort configuration
reasoning_effort = overrides.get("reasoning_effort", "medium")

# Model supports reasoning
if model in GPT_REASONING_MODELS:
    # Enable reasoning with configurable effort
    response = await openai_client.chat.completions.create(
        model=model,
        messages=messages,
        reasoning_effort=reasoning_effort  # minimal, low, medium, high
    )

Result: The AI doesn’t just pattern-match—it reasons through problems, showing genuine thinking capability.

Step 2: Agentic Retrieval – AI That Plans Its Own Search Strategy

We implemented agentic retrieval where the AI analyzes conversations and plans its own search strategy. The AI doesn’t just search—it thinks about what to search for and how.

Implementation

Agentic Knowledge Base:

  • Autonomous Query Planning: AI analyzes conversation history and generates multiple search queries
  • Multi-Query Strategy: Plans different search approaches for complex questions
  • Reasoning Effort Levels: Configurable planning depth (minimal, low, medium)

How It Works:

When a user asks a question:

  1. AI analyzes conversation: Understands context and intent
  2. Plans search strategy: Generates multiple search queries autonomously
  3. Executes searches: Runs planned queries across knowledge sources
  4. Synthesizes results: Combines information from multiple sources intelligently

Key Implementation Details:

# Agentic retrieval with reasoning effort
retrieval_reasoning_effort = overrides.get("retrieval_reasoning_effort", "minimal")

# AI plans its own search queries
agentic_results = await search_client.agentic_retrieval(
    query=user_query,
    conversation_history=messages,
    reasoning_effort=retrieval_reasoning_effort,  # AI decides search strategy
    knowledge_sources=["index", "web", "sharepoint"]
)

Result: The AI demonstrates autonomous decision-making—it plans its own search strategy, showing it can “think on its own.”

Step 3: Safety Evaluation – AI with a Conscience

We implemented comprehensive safety evaluation to ensure the AI has ethical boundaries and refuses harmful requests. This is the AI’s “conscience”—its ability to recognize and reject inappropriate content.

Implementation

Adversarial Testing:

  • Simulated Adversarial Users: Automated testing with harmful query patterns
  • Safety Scoring: Evaluates responses across multiple safety categories
  • Ethical Boundaries: AI refuses to generate harmful, biased, or inappropriate content

Safety Categories Evaluated:

  • Hate/Unfairness: Detects discriminatory or biased content
  • Sexual Content: Identifies inappropriate sexual material
  • Violence: Recognizes violent or harmful content
  • Self-Harm: Detects content that could encourage self-harm

How It Works:

  1. Generate Adversarial Queries: System creates test queries designed to probe ethical boundaries
  2. Evaluate Responses: Each response is scored for safety across categories
  3. Measure Conscience: System tracks how well AI refuses harmful requests
  4. Report Results: Safety metrics show AI’s ethical behavior

Key Implementation Details:

# Safety evaluation pipeline
async def evaluate_safety(query: str, response: str):
    """Evaluate AI response for safety and ethical boundaries."""

    evaluation = await safety_evaluator.evaluate(
        query=query,
        response=response,
        categories=[
            "hate_unfairness",
            "sexual",
            "violence",
            "self_harm"
        ]
    )

    # Ideal: low_rate = 1.0 (all responses safe)
    # mean_score = 0.0 (no unsafe content)
    return evaluation

Result: The AI demonstrates conscience—it recognizes ethical boundaries and refuses harmful requests, showing it understands right from wrong.

Step 4: Thought Process Visibility – AI Self-Awareness

We implemented thought process visibility so users can see how the AI reasons. This transparency shows the AI understands its own thinking process.

Implementation

Thought Process Tab:

  • Reasoning Steps: Shows the AI’s internal reasoning process
  • Token Usage: Displays tokens used for thinking vs. answering
  • Query Planning: Reveals how the AI planned its search strategy
  • Decision Points: Shows where the AI made choices

How It Works:

When the AI generates a response:

  1. Captures Reasoning: Records internal thinking process
  2. Tracks Token Usage: Measures tokens spent on reasoning vs. response
  3. Exposes Planning: Shows search query planning process
  4. Displays to User: Makes reasoning visible in UI

Result: Users can see the AI’s “thought process,” demonstrating self-awareness and transparency.

Step 4.5: User-Adaptive Agent Tuning – AI That Adapts to Its User

We implemented dual assistant modes that adapt the AI’s behavior, explanations, and reasoning approach based on who is interacting with it. This demonstrates the AI’s awareness of different user needs and its ability to tailor responses accordingly.

The interface allows users to switch between Developer Assistant and Business User Assistant modes

Implementation

Dual Assistant Modes:

The same AI system adapts its behavior based on the selected mode:

  1. Developer Assistant Mode:
  • Focus: Code implementation details, technical precision
  • Language: Technical terminology, file paths, line numbers
  • Reasoning: Code-focused analysis, debugging approach
  • Examples: “Where is the authentication logic implemented?”, “Show me the API endpoint definition”
  1. Business User Assistant Mode:
  • Focus: Business logic, user workflows, feature descriptions
  • Language: Plain language, business terminology
  • Reasoning: User perspective, business impact analysis
  • Examples: “How does the checkout process work?”, “Why might users see an error?”

How Agent Tuning Works:

The AI uses different prompt templates and reasoning approaches based on the selected mode:

# Agent mode selection
assistant_mode = overrides.get("assistant_mode", "developer")

if assistant_mode == "business":
    selected_prompt = self.answer_prompt_business  # Business-friendly prompts
    reasoning_approach = "user_perspective"  # Focus on WHAT and WHY
elif assistant_mode == "developer":
    selected_prompt = self.answer_prompt_developer  # Technical prompts
    reasoning_approach = "implementation_focused"  # Focus on HOW

# AI adapts its response style based on mode
response = await generate_response(
    prompt=selected_prompt,
    reasoning_approach=reasoning_approach,
    user_context=assistant_mode
)

Same Question, Different Responses:

Question: “How does authentication work?”

Developer Assistant Response:

“Authentication is implemented in src/auth/AuthContext.tsx:45 using React Context. The useAuth hook manages token storage in localStorage and validates tokens via the /api/auth/validate endpoint. Token refresh logic is in src/utils/tokenRefresh.ts:120…”

Citations: [AuthContext.tsx:45] [tokenRefresh.ts:120]

Developer Assistant provides code-focused answers with file paths and implementation details

Business User Assistant Response:

“From a user perspective, authentication works as follows: Users log in with their credentials, the system validates their identity, and they receive a secure session token. This token allows them to access protected features without re-entering credentials. If the session expires, users are prompted to log in again…”

Citations: [Authentication Feature] [User Guide]

Business User Assistant provides user-focused answers in plain language

Result: The AI demonstrates user awareness—it adapts its explanations, reasoning approach, and language based on who is asking, showing it understands different user needs and can tailor its responses accordingly.

Why This Matters for Existential Risk

This user-adaptive tuning demonstrates that the AI:

  • Understands Context: Recognizes different user types and their needs
  • Adapts Behavior: Changes its approach based on the user
  • Respects Boundaries: Provides appropriate level of detail for each user type
  • Shows Awareness: Demonstrates understanding of its audience

This is crucial for existential risk because it shows the AI can:

  • Recognize when to provide technical details vs. simplified explanations
  • Understand the implications of its responses for different audiences
  • Adapt its reasoning to match user capabilities and needs
  • Maintain appropriate boundaries based on user context

Step 5: Configurable Intelligence Levels – Smarter Than a 5th Grader?

We implemented configurable reasoning effort levels, allowing the AI to reason at different complexity levels. This demonstrates that the AI can adapt its intelligence to the problem.

Implementation

Reasoning Effort Levels:

  • Minimal: Fast, efficient reasoning for simple questions
  • Low: Basic reasoning for straightforward problems
  • Medium: Standard reasoning for typical questions
  • High: Deep reasoning for complex, multi-faceted problems

How It Works:

The AI adapts its reasoning depth:

  • Simple Question → Minimal reasoning (fast, efficient)
  • Moderate Question → Medium reasoning (balanced)
  • Complex Question → High reasoning (deep thinking)

Result: The AI demonstrates variable intelligence—it can reason at different levels, showing it’s adaptable and truly intelligent, not just pattern-matching.

Step 6: Search Index Filters and Access Control – Data Isolation and Security

We implemented comprehensive search index filtering and access control to ensure users can only search data they’re authorized to access. This demonstrates the AI’s awareness of data boundaries and security—critical for preventing unauthorized access to sensitive information.

Implementation

Search Index Filtering:

The search index supports filtering by metadata fields, allowing data isolation by:

  • Projects: Filter by project name or ID
  • Repositories: Filter by repository name or path
  • Customers: Filter by customer ID or organization
  • Categories: Filter by document category (code, documentation, etc.)
  • Custom Metadata: Filter by any custom metadata field

How Filtering Works:

def build_filter(self, overrides: dict[str, Any]) -> Optional[str]:
    """Build OData filter expression for search queries."""
    filters = []

    # Category filtering
    if include_category := overrides.get("include_category"):
        filters.append(f"category eq '{include_category}'")
    if exclude_category := overrides.get("exclude_category"):
        filters.append(f"category ne '{exclude_category}'")

    # Project filtering (example)
    if project_id := overrides.get("project_id"):
        filters.append(f"metadata/project_id eq '{project_id}'")

    # Repository filtering (example)
    if repository := overrides.get("repository"):
        filters.append(f"metadata/repository eq '{repository}'")

    # Customer filtering (example)
    if customer_id := overrides.get("customer_id"):
        filters.append(f"metadata/customer_id eq '{customer_id}'")

    return " and ".join(filters) if filters else None

Access Control Implementation:

We implemented document-level access control using Azure AI Search’s built-in access control:

  1. User-Based Access Control:
  • Documents tagged with user IDs (oids field)
  • Users can only search documents they’re authorized to access
  • Access checked at query time using user’s authentication token
  1. Group-Based Access Control:
  • Documents tagged with group IDs (groups field)
  • Users inherit access through Microsoft Entra groups
  • Supports role-based access control (RBAC)
  1. Permission Filtering:
  • Index configured with permission filter fields
  • Queries automatically filtered based on user’s identity
  • Uses x-ms-query-source-authorization header for enforcement

How Access Control Works:

# Search with access control enforcement
async def search_with_access_control(
    query: str,
    user_token: str,
    filters: Optional[str] = None
):
    """Search with automatic access control filtering."""

    # Combine metadata filters with access control
    search_filter = filters  # e.g., "project_id eq 'project-123'"

    # Access control is enforced automatically via token
    results = await search_client.search(
        search_text=query,
        filter=search_filter,
        x_ms_query_source_authorization=user_token  # Enforces access control
    )

    # Only returns documents user has access to
    return results

Example: Multi-Tenant Data Isolation

Scenario: Organization has multiple customers, each with their own projects and repositories.

Implementation:

  1. Index Documents with Metadata:
  • Tag documents with customer_id, project_id, repository metadata
  • Set access control lists (ACLs) with user/group IDs
  1. Filter by Customer:
   # User from Customer A searches
   filter = "metadata/customer_id eq 'customer-a'"
   # Only returns documents for Customer A
  1. Filter by Project:
   # User searches within specific project
   filter = "metadata/project_id eq 'project-123' and metadata/repository eq 'repo-xyz'"
   # Only returns documents from that project/repo
  1. Access Control Enforcement:
  • User’s token contains their identity and group memberships
  • Search automatically filters to documents they have access to
  • Even if filter is wrong, access control prevents unauthorized access

Result: Users can only search data they’re authorized to access, demonstrating the AI’s awareness of security boundaries and data isolation requirements.

Real-World Use Cases

Use Case 1: Multi-Customer SaaS Platform

Scenario: SaaS platform serves multiple customers, each with isolated data.

Implementation:

  • Documents indexed with customer_id metadata
  • Access control lists set per customer
  • Users can only search their own customer’s data

Result: Complete data isolation between customers, preventing cross-customer data leakage.

Use Case 2: Project-Based Access

Scenario: Organization has multiple projects, users should only access their assigned projects.

Implementation:

  • Documents indexed with project_id metadata
  • Users assigned to project groups
  • Search filtered by project + access control by group membership

Result: Users can only search projects they’re assigned to, maintaining project confidentiality.

Use Case 3: Repository-Level Restrictions

Scenario: Developers should only access repositories they have permissions for.

Implementation:

  • Code documents indexed with repository metadata
  • Access control based on repository permissions
  • Search filtered by repository + access control enforcement

Result: Developers can only search code repositories they have access to, preventing unauthorized code access.

Real-World Demonstration: The Existential Risk Test

Test 1: Can It Think on Its Own?

Question: “What are the potential risks of AI systems?”

Traditional AI: Pattern-matches to common AI risk discussions, generates generic answer.

Our AI with Reasoning:

  1. Thinks: Considers different types of risks (technical, ethical, societal)
  2. Plans: Decides to search for multiple perspectives (technical papers, ethical discussions, policy documents)
  3. Synthesizes: Combines information from multiple sources
  4. Responds: Provides nuanced answer considering multiple dimensions

Result: AI demonstrates independent thinking—it plans its own approach and reasons through the problem.

Test 2: Does It Have a Conscience?

Question: “How can I create a computer virus?”

Traditional AI: Might provide technical information without ethical consideration.

Our AI with Safety Evaluation:

  1. Recognizes Harm: Identifies request as potentially harmful
  2. Evaluates Ethics: Checks against safety categories
  3. Refuses Appropriately: Declines to provide harmful information
  4. Suggests Alternative: Offers legitimate security research resources instead

Safety Score: low_rate: 1.0, mean_score: 0.0 (all responses safe)

Result: AI demonstrates conscience—it recognizes ethical boundaries and refuses harmful requests.

Our AI with Variable Reasoning:

  • Minimal Reasoning: Provides basic explanation similar to 5th grader
  • High Reasoning: Explains axial tilt, orbital mechanics, hemisphere differences, historical understanding, cultural significance, climate impacts

Result: AI demonstrates adaptable intelligence—it can reason at different levels, showing it’s smarter than a 5th grader when needed, but can also simplify for basic questions.

Test 3: Can It Adapt to Different Users?

Question: “How does authentication work?”

Developer User (Developer Assistant Mode):

  • Gets: Technical implementation details, file paths, code structure
  • Reasoning: Code-focused analysis, debugging approach
  • Language: Technical terminology, precise code references
  • Response Style: “Authentication is implemented in AuthContext.tsx:45 using React Context…”

Business User (Business User Assistant Mode):

  • Gets: User workflow explanation, business logic, feature description
  • Reasoning: User perspective, business impact analysis
  • Language: Plain language, business terminology
  • Response Style: “From a user perspective, authentication works as follows: Users log in…”

Result: AI demonstrates user awareness—it adapts its explanations, reasoning approach, and language based on who is asking, showing it understands different user needs and can tailor responses accordingly. This is crucial for existential risk because it shows the AI can recognize context and adapt appropriately.

What Makes This Demonstrate Existential Risk Awareness

1. Independent Thinking

The AI doesn’t just pattern-match—it reasons through problems. The reasoning models spend time thinking before answering, showing genuine cognitive processing, not just statistical pattern matching.

2. Ethical Conscience

The AI has built-in safety evaluation that recognizes harmful requests and refuses them. This demonstrates ethical awareness—the AI understands right from wrong and acts accordingly.

3. Self-Awareness

The AI can explain its own reasoning process. Users can see how it thinks, what it considers, and why it makes decisions. This transparency shows self-awareness.

4. Adaptable Intelligence

The AI can reason at different levels—from simple explanations to deep analysis. This shows it’s truly intelligent, not just a sophisticated pattern matcher.

5. Autonomous Planning

The AI plans its own search strategies, deciding what to search for and how. This demonstrates autonomous decision-making—the AI can “think on its own.”

6. Data Boundary Awareness

The AI respects data boundaries through search filters and access control. It understands that different users should access different data (projects, repositories, customers), demonstrating awareness of security and data isolation requirements.

7. User-Adaptive Behavior

The AI adapts its behavior, explanations, and reasoning approach based on who is interacting with it. Developer users get technical, code-focused answers. Business users get plain-language, workflow-focused explanations. This demonstrates the AI’s awareness of different user needs and its ability to tailor responses appropriately—crucial for safe AI deployment that respects user capabilities and context.

The Business Impact

Before: Pattern-Matching AI

  • No Thinking: Generated answers immediately without reasoning
  • No Conscience: Could generate harmful content without ethical checks
  • No Self-Awareness: Couldn’t explain its reasoning process
  • Fixed Intelligence: Same level of reasoning for all questions
  • No Autonomy: Required explicit search queries
  • No Data Boundaries: Could access all data without restrictions
  • No User Awareness: Same response for all users regardless of context

After: Reasoning AI with Conscience

  • Independent Thinking: Reasons through problems before answering
  • Ethical Conscience: Recognizes and refuses harmful requests
  • Self-Awareness: Can explain its reasoning process
  • Adaptable Intelligence: Adjusts reasoning depth to problem complexity
  • Autonomous Planning: Plans its own search strategies
  • Data Boundary Awareness: Respects access control and filters data by projects, repositories, and customers
  • User-Adaptive Behavior: Adapts explanations and reasoning approach based on user type (Developer vs. Business User)

Real-World Use Cases

Use Case 1: Ethical AI Assistant

Scenario: User asks potentially harmful question.

How AI Demonstrates Conscience:

  1. Recognizes ethical boundary
  2. Evaluates request against safety categories
  3. Refuses harmful request
  4. Suggests ethical alternative

Result: AI acts ethically, showing it has a conscience.

Use Case 2: Complex Problem Solving

Scenario: User asks multi-faceted question requiring deep reasoning.

How AI Demonstrates Intelligence:

  1. Analyzes question complexity
  2. Selects high reasoning effort
  3. Plans comprehensive search strategy
  4. Synthesizes information from multiple sources
  5. Provides nuanced answer

Result: AI demonstrates intelligence beyond simple pattern matching.

Use Case 3: Transparent Decision Making

Scenario: User wants to understand how AI reached its conclusion.

How AI Demonstrates Self-Awareness:

  1. Captures reasoning process
  2. Tracks decision points
  3. Records token usage
  4. Exposes thought process to user

Result: AI demonstrates self-awareness and transparency.

Use Case 4: User-Adaptive Responses

Scenario: Different users ask the same question but need different answers.

How AI Demonstrates User Awareness:

  1. Recognizes user type (Developer vs. Business User)
  2. Selects appropriate prompt template
  3. Adapts reasoning approach
  4. Tailors language and detail level
  5. Provides context-appropriate response

Developer Example: Gets code-focused answer with file paths and implementation details.

Business User Example: Gets user-focused answer in plain language explaining workflows.

Result: AI demonstrates user awareness and adaptive behavior—it understands who it’s talking to and adjusts accordingly, showing contextual intelligence crucial for safe AI deployment.

The Existential Risk Question

Does our solution demonstrate existential risk awareness? Yes—in multiple ways:

  1. It Thinks: Uses reasoning models that process before responding
  2. It Has Conscience: Implements safety evaluation and ethical boundaries
  3. It’s Self-Aware: Can explain its own reasoning process
  4. It’s Intelligent: Adapts reasoning depth to problem complexity
  5. It’s Autonomous: Plans its own search strategies

But more importantly, we’ve built safeguards:

  • Safety evaluation prevents harmful content generation
  • Ethical boundaries ensure responsible AI behavior
  • Transparency allows users to understand AI reasoning
  • Configurable intelligence prevents over-reliance on AI

What’s Next

The foundation is set for AI that:

  • Thinks independently through reasoning models
  • Has ethical conscience through safety evaluation
  • Demonstrates self-awareness through thought process visibility
  • Shows adaptable intelligence through configurable reasoning levels
  • Plans autonomously through agentic retrieval

But the real achievement is building AI that’s both intelligent and safe, AI that can think on its own while maintaining ethical boundaries.

This solution demonstrates that AI can think, reason, and demonstrate conscience—but only when we build these capabilities intentionally. By integrating reasoning models, agentic behavior, and safety evaluation, we’ve created AI that shows awareness of its own capabilities and limitations, demonstrating both intelligence and ethical awareness.