> For the complete documentation index, see [llms.txt](https://isubasinghe.gitbook.io/isithas-wiki/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://isubasinghe.gitbook.io/isithas-wiki/computer_science/distributed_systems/distributed_systems_challenges.md).

# Challenges of Distributed Systems

## Basic idea

Distributed systems must overcome a fixed set of recurring hardships beyond what single-machine programs face: heterogeneity, openness, security, scalability, failure handling, concurrency, and transparency.

## Key facts

* The "eight fallacies of distributed computing": network is reliable / zero latency / infinite bandwidth / secure / topology stable / one admin / zero transport cost / homogeneous — all false.
* Transparency dimensions: access, location, concurrency, replication, failure, mobility, performance, scaling.
* Failure-handling toolkit: detect, mask, tolerate, recover, redundancy.
* Heterogenity
* Openness
* Security
* Scalability
* Failure Handling
* Concurrency
* Transparency

## Heterogenity

Distributed Systems may use completely different hardware and software. These listed above are examples of differring hardware/software:

* Networks
* Computer Hardware
* Operating Systems
* Programming Languages
* Implementation by different developers

  These problems have some solutions such as:
* Using standard protocols
* Using agreed upon message formats and data types
* Adhering to an API
* Middleware
* Portable code (code that is not tighly coupled to specific hardware)

## Middleware

Middleware is a software layer between the distributed application and the operating system that:

* provides a programming abstraction and
* masks the heterogenity of underlying platform

### Middleware Models

* Distributed File Systems
* Remote Procedure Call (RPC)
* Remote Method Invocation (RMI)
* Distributed Documents
* Distributed Databases

## Openness

Openness refers to the ability to extend the system in different ways by adding hardware or software resources. Following are some approaches to address openness:

* Publishing key interfaces
* Allowing a uniform communication mechanism to communicate over the published interfaces.
* Ensuring all implementations adhere to the published standards.

## Security

There are three aspects of security:

1. Confidentiality
2. Integrity
3. Availability

Security Mechanisms:

1. Encryption
2. Authentication
3. Authorization

## Scalability

A system should be able to handle growth in users.

### Scalability Challenges

* Cost of physical resources
* Controlling performance loss
* Resources should not run out
* Avoiding performance bottlenecks

## Failure Handling

* **Detecting :** some types of failures can be detected and some are hard to be certain about.
* **Masking :** some failures that have been detected can be hidden or made less severe;
* **Tolerating :** it is sometimes impractical to try and handle every failure that occurs
* **Recovery :** some failures can recover, for example a rollback mechanism
* **Redundancy :** some failures can be made to tolerate failure using redundant components (eg multiple services that provide the same service(failover))

## Concurrency

* Multiple clients can access the same resource at the same time, in some cases for updates.
* One approach to handling concrrency is making access sequential - slows down the system.
* Semaphores supported by the operating system is a well accepted mechanism to handle concurrency.

## Transparency

Hiding certain aspects from the user and application programmer

* Access
* Location
* Concurrency
* Replication
* Failure
* Mobility
* Performance
* Scaling


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://isubasinghe.gitbook.io/isithas-wiki/computer_science/distributed_systems/distributed_systems_challenges.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.