Assignment Submission

Course: Artificial Intelligence (Course Code - PGS-AI-205)

Topic: Question 3 — Probabilistic Reasoning in Artificial Intelligence

---

Introduction

In real-world environments, Artificial Intelligence systems rarely operate under conditions of complete certainty or perfect information. Agents frequently face noisy sensor data, incomplete domain knowledge, non-deterministic action outcomes, and unpredictable environments. While classical logic operates strictly on binary truth values (True/False), probabilistic reasoning enables AI agents to quantify, represent, and reason under uncertainty using the mathematical framework of probability theory.

1. Concepts of Probability Theory

Probability theory provides the mathematical foundations for managing uncertain knowledge in AI systems.

Key Concepts

• Random Variables: Variables representing uncertain events or states in the domain. They can be discrete (e.g., Weather ∈ {Sunny, Rainy, Cloudy}) or continuous (e.g., Temperature ∈ ℜ).
• Prior (Unconditional) Probability: P(A) represents the baseline degree of belief in proposition A in the absence of any surrounding evidence.
• Conditional (Posterior) Probability: P(A | B) represents the updated probability of event A given that evidence B has been observed:
  P(A | B) = P(A ∧ B) / P(B)
• Joint Probability Distribution: A complete table assigning probabilities to every possible combination of values across all random variables in a domain, denoted as P(X₁, X₂, ..., X_n). While a full joint distribution allows answering any probabilistic query, its size grows exponentially as O(dⁿ), where d is domain size and n is the number of variables.
• Axioms of Probability (Kolmogorov’s Axioms):
  1. 0 ≤ P(A) ≤ 1 for any event A.
  2. P(Ω) = 1, where Ω is the entire sample space.
  3. For mutually exclusive events A and B, P(A ∨ B) = P(A) + P(B).

2. Bayes' Theorem

Bayes' Theorem is the fundamental mathematical mechanism used in AI to update belief states in light of new evidence.

Formula

Components

• P(H | E) (Posterior Probability): The updated probability of hypothesis H after observing evidence E.
• P(E | H) (Likelihood): The probability that evidence E would be observed if hypothesis H were true.
• P(H) (Prior Probability): The initial probability of hypothesis H before receiving evidence E.
• P(E) (Marginal Likelihood / Evidence): The total probability of observing evidence E across all hypotheses, calculated as ∑ P(E | H=h) · P(H=h).

Example Application

If a diagnostic medical test for a rare condition (D) has an accuracy rate where P(Positive | D) = 0.99, and the disease prior prevalence is P(D) = 0.001, Bayes' Theorem prevents false positives from misleading clinicians by calculating the actual posterior probability P(D | Positive).

3. Bayesian Networks

A Bayesian Network (also called a Belief Network or Directed Acyclic Graph) is a compact, probabilistic graphical model that represents conditional dependencies among a set of random variables.

Structure & Components

• Nodes: Represent random variables in the domain.
• Directed Edges: Represent direct causal or conditional influences between variables. An edge from node A to B indicates that A is a parent of B.
• Conditional Probability Tables (CPTs): Every node X_i maintains a CPT that specifies the probability distribution P(X_i | Parents(X_i)) given every possible assignment of its parents.

Factorization of the Joint Distribution

Instead of storing a massive full joint probability table, a Bayesian Network exploits conditional independence to factorize the joint probability as the product of local conditional distributions:

P(X₁, X₂, ..., X_n) = ∏ P(X_i | Parents(X_i))

The Markov Blanket

A node in a Bayesian Network is conditionally independent of all other nodes in the network given its Markov Blanket, which consists of its parents, its children, and its children's other parents.

4. Inference Methods in Bayesian Networks

Inference in Bayesian Networks involves calculating the posterior probability distribution over a set of query variables given observed evidence variables.

A. Exact Inference Methods

• Enumeration Algorithm: Computes queries by summing out hidden variables from the joint distribution. While conceptually simple, it has exponential time complexity O(dⁿ).
• Variable Elimination: An optimized dynamic programming technique that avoids repeated calculations by evaluating terms from right to left and caching intermediate factors.
• Junction Tree Algorithm: Transforms the network graph into a tree of cliques, allowing exact message-passing algorithms (similar to belief propagation) in O(n) time relative to tree-width.

B. Approximate Inference Methods

Since exact inference in arbitrary Bayesian Networks is NP-hard, large real-world applications rely on stochastic approximation algorithms:

• Direct / Rejection Sampling: Generates millions of sample states from the network according to prior probabilities and rejects any sample that contradicts the observed evidence.
• Likelihood Weighting: Fixes evidence variables to their observed values and assigns a weight to each sample based on the likelihood of the evidence given its parents, avoiding wasted samples.
• Markov Chain Monte Carlo (MCMC / Gibbs Sampling): Generates successive samples by making local random changes to one variable at a time, conditioned on the current values of its Markov Blanket.

5. Probabilistic Reasoning vs. Logical Reasoning

6. Applications of Probabilistic Reasoning

A. Healthcare & Clinical Diagnostics

In medical decision-support systems, Bayesian Networks model complex physiological relationships. For example, a diagnostic system can calculate the probability of a patient having lung cancer given various symptoms (e.g., shortness of breath, chronic cough) and risk factors (e.g., smoking history, age), helping doctors decide which additional tests are cost-effective.

B. Finance & Credit Risk Management

Financial institutions use Bayesian models for credit scoring, algorithmic trading, and automated fraud detection. A fraud detection system evaluates incoming transactions in real time: P(Fraud | Amount, Location, Device). If a high-value transaction occurs in an unusual location from an unrecognized IP address, the network updates the posterior probability of fraud and automatically triggers a hold on the account.

C. Decision Support Systems & Autonomous Perception

• Spam Filtering: Naïve Bayes Classifiers calculate the probability that an incoming email is spam based on word occurrence frequencies (P(Spam | words)).
• Autonomous Vehicles: Self-driving cars use recursive Bayesian estimation filters (such as Kalman Filters and Particle Filters) to fuse noisy radar, LiDAR, and camera streams to continuously estimate the exact position and velocity of surrounding vehicles and pedestrians.

Conclusion

Probabilistic reasoning expands the capabilities of Artificial Intelligence beyond rigid binary logic. By combining probability theory, Bayes' Theorem, and structured models like Bayesian Networks, AI systems can process incomplete or noisy data, make sound predictions, and execute rational decisions in uncertain real-world environments.