Internet evolution

Internet Of Things (IoT)

Internet history timeline

• 1961 – First packet-switching papers
• 1966 – Merit Network founded
• 1969 – ARPANET carries its first packets
• 1980 – Ethernet standard introduced
• 1982 – TCP/IP, SMTP
• 1983 – Domain Name System (DNS)
• 1985 – First .COM domain name registered
• 1988 – OSI Reference Model released
• 1990 – ARPANET decommissioned
• 1990 – IMDb Internet movie database
• 1991 – World Wide Web (WWW)
• 1994 – Full text web search engines
• 1995 – IPv6 proposed
• 1998 – Google Search
• 2001 – New top-level domain names activated
• 2003 – Skype Internet voice calls
• 2004 – Facebook social networking site
• 2009 – Bing search engine
• 2010 – Instagram

Internet evolution: 1969

• United States Department of Defense
• DARPA
• Universities
• ARPANET (Advanced Research Projects Agency Network)
• 1st September - first server
• Honeywell DP-516
• 12/24 Kb RAM

Internet evolution: < 1971

• 29th of October 1969 - first communication between computers ("LOG" message)
• first email application (POP/IMAP protocol)
• network used for sending emails
• first mailing list, newsgroups, forums
• actively evolve protocols

Internet evolution: 1973-1974

• UK & Norway
• international network
• Internet Transmission Control Program

Internet evolution: 1983

• NCP > TCP/IP
• "Internet" term

Internet evolution: 1984

• Domain Name System
• NSFNet

Internet evolution: 1988

• IRC (Internet Relay Chat)

Internet evolution: 1990

• NSFNet owned
• first Dial-up access

Internet evolution: 1991

• NCSA Mosaic (till 1997)
• http protocol, HTML, URI (Tim Berners-Li)

Internet evolution: 1991

• NCSA Mosaic 1992

Internet evolution: 1992

• HTML1

Internet evolution: 1994

• internet providers
• WWW Consorcium, W3C
• HTML2
• 10kk PCs, 30k web-sites

Internet evolution: 1995

• Netscape Navigator
• Internet Explorer

Internet evolution: 1998

• Pope John Paul II founded "International Day for the Internet" (4th of March / 14th of December)
• IE4, Netscape Navigator 4, IE5 beta (with javascript)
• Web Standards Project
• CSS Samurai

Internet evolution: 2000

• HTML4
• XHTML 1.0
• IE5 for Macintosh
• 327kk of users (100kk in USA)

Internet evolution: 2004

• Flash Lite 1.0
• 700kk of users

Internet evolution: 2011

• International Space Station
• 2kkk of users

Communication evolution

Communication evolution

Communication evolution

Communication evolution

Communication evolution

Communication evolution

Communication per-Computers

Request-Response Model (request–reply, synchronous)

1. establishing a connection
2. data exchange
3. end of session

web browser

frontend


html / css / js / etc

web server

backend

database / server / data ops / cache / etc

Communications protocol

describes how devices can communicate,
and defines the format of data beign exchanged

Real life example of protocol

What is a Protocol?

Protocol examples

• Internet Protocol
  set of rules to send and receive messages at the Internet address level
• File Transfer Protocol
  sets of rules to use with corresponding programs elsewhere on the Internet
• Simple Mail Transfer Protocol
• Secure Shell

Real life example of layered architecture

The pen

Real life example of layered architecture

The pen in box

Real life example of layered architecture

The pen in box with declaration

Real life example of layered architecture

The pen in box with declaration is delivering to receiver

Real life example of layered architecture

The pen in box with declaration is delivered to receiver.

He got it.

Real life example of layered architecture

The pen in box with declaration is delivered to receiver.
He got it, confirmed that he is receiver and got the box.

Real life example of layered architecture

The pen in box with declaration is delivered to receiver.
He got it & confirmed personality, unboxed and got the pen.

Real life example of layered architecture

Internet layers

Internet layers

Internet layers

TCP and UDP protocols

TCP - Connection-oriented (3-way handshake)
UDP - Connectionless

TCP/IP Model

TCP - Connection-oriented (3-way handshake)
UDP - Connectionless

TCP/IP Model

TCP/IP Model: Application layer

• units: data
• protocols: FTP, HTTP, POP3, IMAP, telnet, SMTP, DNS, TFTP, ...

TCP/IP Model: Transport layer

• units: segments
• protocols: TCP, UDP, etc

TCP/IP Model: Internet layer

• units: packet
• protocols: IP, ICMP, ARP, RARP, etc

TCP/IP Model: Link layer

• units: bit
• protocols: Ethernet, Token Ring, FDDI, X.25, Frame Relay, RS-232, v.35, etc

OSI Model

All People Seem To Need Data Processing

OSI Model: Application Layer

• unit: data
• Remote login to hosts: Telnet
• File transfer: File Transfer Protocol (FTP), Trivial File Transfer Protocol (TFTP)
• Electronic mail transport: Simple Mail Transfer Protocol (SMTP)
• Networking support: Domain Name System (DNS)
• ...

OSI Model: Presentation Layer

• unit: data
• ASCII, Unicode

OSI Model: Session Layer

• unit: data
• SSL, TLS, SOCKS

OSI Model: Transport Layer

• TCP, UDP, SMB

OSI Model: Network Layer

• unit: segments
• IPv4, IPv6, TCP/IP

OSI Model: Datalink Layer

• unit: packet
• Ethernet, PPP, PPPoE, PPTP

OSI Model: Physical Layer

• unit: images

IP Address

IPv4

Internet Protocol version 4

IPv4 reserved

IPv6

Internet Protocol version 6

2001:0db8:11a3:09d7:1f34:8a2e:07a0:765d
http://[2001:0db8:11a3:09d7:1f34:8a2e:07a0:765d]:8080/
0000:0000:0000:0000:0000:0000:ae21:ad12 => ::ae21:ad12

IPv6 reserved

Port

16-bit number
google "well-known ports"

Роскомнадзор

The Golden Shield Project

NAT

Network Address Translation
IP Masquerading, Network Masquerading и Native Address Translation

DNS

Domain Name System

Time To Live

lowest = 5 min
default = 24 hours

Types of records

A (map IP to Host)
CNAME (alias with different A record)
MX (for sending an email)

Reverse Lookup

Lookup Hosts on specific IP

DNS request

➜ ✗ host cursor.education
cursor.education has address 91.194.250.152
cursor.education mail is handled by 10 mx.yandex.ru.
              

DNS request (detailed)

➜ ✗ host -a cursor.education
Trying "cursor.education"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43851
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0

;; ANSWER SECTION:
cursor.education.	14399	IN	SOA	ns7.rehost.com.ua. hostmaster.redo.com.ua. 2016021502 10200 3600 1209600 86400
cursor.education.	14399	IN	NS	ns8.rehost.com.ua.
cursor.education.	14399	IN	NS	ns7.rehost.com.ua.
cursor.education.	14399	IN	A	91.194.250.152
cursor.education.	14399	IN	TXT	"v=spf1 a mx ip4:91.194.250.152 ~all"
cursor.education.	14399	IN	MX	10 mx.yandex.ru.

Received 227 bytes from 8.8.8.8#53 in 128 ms
              

DNS query lookup

DNS resource records (RR)

• https://dnsquery.org/
• http://ping.eu/nslookup/
• https://www.ultratools.com/tools/dnsLookup

DNS /etc/hosts

Sockets in real life

Socket = IP + Port

Socket = IP + Port

URL

Uniform Resource Locator

URI

Uniform Resource Identifier

Relative vs Absolute URLs

https://www.example.com

Absolute URLs:

• http://yourdomain.com/images/example.png
• //yourdomain.com/images/example.png

• /images/example.png
• images/example.png
• file.php
• ../.././../css/default.css

Trace Route

➜  ✗ traceroute cursor.education
traceroute to cursor.education (91.194.250.152), 64 hops max, 52 byte packets
 1  192.168.0.1 (192.168.0.1)  1.946 ms  5.349 ms  0.761 ms
 2  77-123-168-1-ptr.volia-lviv.com (77.123.168.1)  17.323 ms  13.195 ms  37.056 ms
 3  unknown.volia-lviv.com (77.120.32.202)  12.616 ms  11.017 ms  11.824 ms
 4  v973.tengig3-2.diamond.volia.net (77.120.0.73)  32.960 ms  21.498 ms  39.309 ms
 5  * * *
 6  176.241.108.214 (176.241.108.214)  18.866 ms  19.270 ms  19.739 ms
 7  * * *
 8  dd.rehost.com.ua (91.194.250.152)  22.207 ms  20.224 ms  22.833 ms
              

TErminaL NETwork

Telnet SMTH

$ telnet smtp.gmail.com 587
EHLO localhost
STARTTLS
MAIL FROM: <support@gmail.com>
              

HTTP Protocol

HyperText Transfer Protocol

http over telnet

$ telnet google.com 80

GET / HTTP/1.1
Host: google.com
              

http: accessing resources

                 <protocol>://<server>/<path>
              

http: characteristics

• request-response mechanism
  client sending a request to server,
  server generates a response
• resource identification
  each http request includes a URI
• statelessness
  the server does not maintail any information about the transaction
• meta data support
  metadata about information can be exchanged in the message

http: request message/format

(request line)  GET /index.html HTTP/1.0
(headers lines) Host: www.site.com
                Date: 123
                User-Agent: Mozilla/1.0 (Chrome v0.5)
                Accept-Language: ru, en
                *Content-Length: message-payload
(carriage line feed)
                (message body)
              

(method) (resource-url) HTTP/(version)

GET /index.html HTTP/1.0
GET http://ya.ru/index.html HTTP/1.1
POST http://olx.ua/post HTTP/1.1
HEAD /ping HTTP/1.0
              

http: request methods

• GET
  to retreive whatever information
• POST
  used to send data to the server for updates
  submit information to web server

http: request method example

GET /wiki/HTTP HTTP/1.0
Host: ru.wikipedia.org
User-Agent: HTTPTool/1.0
              

http: other request methods

• HEAD
  response does not include message body
  useful for getting metadata without transfering the resource
• PUT
  creates the resource if it does not create
• DELETE
  removes the web object
• TRACE
  invokes a remove application layer feedback of the request message
  useful for testing what is being received at the server
• OPTIONS
  requests information about communication options available to server

safe/idempotent methods

idempotent: a=4;
non-idempotent: a++;

http: response format

(status line)  HTTP/1.0 200 OK
(header lines) Data: 123
               Server: Apache/1.3.12 (Unix)
               Last-Modified: (data)
               Content-Type: text/html
               Content-Length: (message-payload)
(carriage line feed)
               (message body)
              

HTTP/(version) (status-code) (status-message)

HTTP/1.0 200 OK
HTTP/1.1 403 Forbidden
HTTP/1.1 404 Not Found
              

http: result code and phrase

1xx: Informational - not done yet
2xx: Success - you win
3xx: Redirection - you lose but try again
4xx: Client Error - you lose, your fault
5xx: Server Error - you lose, my bad
              

• 200 OK
• 301 Moved Permanently
• 302 Found
• 307 Temporary Redirect
• 401 Unauthorized
• 403 Forbidden
• 404 Not Found
• 500 Internal Server Error

Redirection

http allows servers to redirect a client request to a different location

HTTP/1.1 302 Found
Location: ...
              

improvements in http/1.1

• persistent connections
  keeps the connection open after the server response
• request pipelining
  allow a client to send several requests without waiting for a response
• chunked encoding
  allows sender to break a message into arbitrary sized chunks
  useful for dynamically created response messages

Реп'яшки

Session cookies

HTTP Setting Cookies

Name, Value, Expires, Path, Domain

HTTP Caching

Cache-Control, Last-Modified and Expires

HTTP Caching

Cache-Control: no-cache
Pragma: no-cache

Cache-Control: private/public
Cache-Control: max-age=31536000

Last-Modified: Wed, 25 Feb 2015 12:00:00 GMT
Expires: Thu, 25 Feb 2016 12:00:00 GMT

If-Modified-Since: Wed, 25 Feb 2015 17:42:04 GMT
              

HTTP Compression

Accept-Encoding: gzip, deflate
Content-Encoding: gzip
Content-Length: 1253
              

HTTP Basic Authentication

Response:

HTTP/1.1 401 Access Denied
WWW-Authenticate: Basic realm="My Server"
Content-Length: 0
              

GET /securefiles/ HTTP/1.1
Host: www.httpwatch.com
Authorization: Basic aHR0cHdhdGNoOmY=
              

HTTPS = Encrypted HTTP

HTTP cons

HTTPS principle

Man-in-the-middle attack

When you need https?

• online store (accept online stores, credit cards, etc
• login & signin pages
• stores personal user data
• company with privacy & security requirements

HTTP vs HTTPs

FTP & sFTP & TFTP

port = 21
active & passive

SSH

Secury Shell, remote console access
Auth by PKI, credentials
PuTTY

RDP

Remote Desktop Protocol, for remote graphical access

Google.com, and hit enter!

So, what are the steps to get the Google's page?
connection, DNS, HTTP request, HTTP response, parse URL, render the page?

• parsing of URL (detected HTTP protocol, and "/" resource)
• DNS request to get a IP
• open Socket connection over TCP (ip + port)
• create HTTP packet & send
• wait & get the HTTP response
• close the TCP connection
• parse & render the response body in browser

Sniffers / Analyzers

Fiddler

Chrome/Webkit inspector

Postman

http://www.getpostman.com/
https://echo.getpostman.com/

REST

(draw restful approach, frontend & backend separation via API)

REpresentational State Transfer

Resource naming:

Swagger

google "swagger online example"

Real-time apps

• real-time (event-driven) apps
• social & online games
• collaborative platforms
• financial applications
• social feeds
• chats
• etc

Real-time in HTTP (Polling Architecture)

Request every n seconds

Real-time in HTTP (Long-Polling Architecture)

Request every n seconds \w wait-delay

Real-time in HTTP (HTTP Streaming)

WebSockets

WebSockets architecture

WebSockets pros & cons

• full-duplex (B-directional)
• single TCP connection
• only 2 bytes of overhead

WebSockets support in browsers

caniuse.com 89.42%

WebSockets Example

tbd

WebSockets vs. HTTP

CDN

Content Delivery Network

Subresource Integrity

https://www.srihash.org/