Understanding the Internals of PostgreSQL

Author: clarklindev

03.postgreSQL.md

@@ -146,4 +146,138 @@ CREATE TABLE followers (
 	follower_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
 	UNIQUE(leader_id, follower_id)
 );
-```
+```
+
+#### restoring database from a backup
+sql databases backup to a .sql file.
+
+restore from backup:
+pgadmin -> servers-> localhost -> databases -> instagram -> *(right click) -> restore
+filename -> ... -> *(format -> all files or .sql) -> *(select file to restore from)
+
+might have to set the 'PostgreSQL Binary Path':
+In pgAdmin select File -> Preferences and look for Path and then click on Binary Path and it needs your path where it says PostgreSQL Binary Path. Go to your computer -> C: (on windows) -> Program Files -> PostgreSQL -> your version -> bin 
+
+restore options tab -> enable: 
+	1. types of objects -> only data -> yes
+	2. do not save -> owner -> yes
+	3. queries -> single transaction -> yes
+	4. disable -> trigger -> yes
+	5. miscelleaneous / behavior -> verbose messages -> yes
+
+#### restore database from scratch 
+steps to restore:
+1. close all query tools windows
+2. stop all server activity except the first under pgadmin -> dashboard
+3. drop the database
+4. create new database
+5. restore from backup (same step as above) BUT restore options tab -> enable ONLY 4 steps from above
+
+restore from backup:
+pgadmin -> servers-> localhost -> databases -> instagram -> *(right click) -> restore
+filename -> ... -> *(format -> all files or .sql) -> *(select file to restore from)
+
+1. types of objects -> only data -> NO
+
+2. do not save -> owner -> yes
+3. queries -> single transaction -> yes
+4. disable -> trigger -> yes
+5. miscelleaneous / behavior -> verbose messages -> yes
+
+#### where postgreSQL stores data
+```SQL
+show data_directory;
+```
+this gives us a path like: /users/*username*/library/application support/postgres/var-12
+all database content is stored in 'base' folder
+
+#### Folder where database is stored
+results are table(oid, datname).
+the oid is the internal identifier (which is also the folder name inside the 'base' folder) that is associated with a database
+```SQL
+SELECT oid, datname
+FROM pg_database;
+```
+
+#### file where database table is stored
+inside that folder, each file represents one object inside database
+table (oid, relname, relnamespace, reltype ...etc)
+```SQL
+SELECT * FROM pg_class;
+```
+each row represents an object refers to eg. tables, indexes, primary keys.
+so find the table we want and look at associated 'oid', 
+that will be name of file which will contain all data for that object. eg. 22445
+
+* Heap file - file that contains all the data (rows) of our table
+* page or block - heap is divided into many different 'blocks' or 'pages'. Each page/block stores a number of rows
+* tuple or item - individual row from the table.
+
+Heap file is divided up into many page/blocks and each page/block has many tuple/item/rows.
+each block is 8kb.
+
+* Block is binary data (stores 0's and 1's) and divided up into parts. 
+* begining part stores information about block, 
+* then next few blocks stores information about rows, where to find data.
+* free space
+* actual data for tuples.
+
+#### deciphering a page/block
+
+[postgreSQL docs about page/block](http://postgresql.org/docs/current/storage-page-layout.html)
+* Table 68.2 Overall Page layout
+
+* view with hex reader (vs code extensions - microsoft hex editor)
+* open the file with the hex editor (CTRL+SHIFT+P) -> hex editor
+* hex code is actual "shortcut" or encoded binary 1's and 0's
+* each hex is 1 byte.
+
+#### Page Layout
+* PageHeaderData
+
+##### PageHeaderData 
+24 bytes long. contains information about page
+Table 68.3 PageHeaderData layout
+* pd_lsn - 8 bytes
+* pd_checksum - 2 bytes
+* pd_flags - 2 bytes
+* pd_lower - 2 bytes (*offset to start of free space)
+* pd_upper - 2 bytes (*offset to end of free space)
+* pd_special - 2 bytes
+* pd_pagesize_version
+* pd_prune_xid
+
+pd_lower:
+Once we know where pd_lower is located - we know the number of bytes starting at beginning of page to start off free space.
+clicking on first byte -> then in hex editor (Data inspector) - pd_lower - Int16 value -> eg. 228
+228 means we count off 228 bytes from start of page/block (228 / 16 (each row has 16) = 14.25), next value is the start of free space
+
+pd_higher:
+2 bytes - clicking on first byte -> data inspector -> int16 -> value is 296.
+296 gives value for end of free space area -> 296/16(each row in hex editor has 16 cols) = 18.5
+counting down from begining of block to 18.5, we will find the end of free space.
+
+##### ItemIdData
+4 bytes per item. array of identifiers pointing to actual items. 
+There are multiple of these after the page header.
+
+how data is stored:
+click on start of 4 bytes, then go to the next byte (2nd of 4 bytes).
+Data Inspector -> look at 8 bit binary eg. 10110100
+look at the 2nd digit of the 8bit binary.
+find binary to decimal converter: 0110100
+then go back and copy the entire 8 bit binary 10110100
+
+so in our example, concat 0110100 + 10110100 and convert to decimal = eg. 296 
+this will give you a decimal number which is the number of bytes from the start of the page to the first item.
+296 / 16 = 18.5 rows. and that will be the start of the first item.
+
+
+##### Freespace - unlocated space.
+
+##### Items - actual items themselves. 
+to calculate length of data. click on 3rd byte of 4 the bytes. look at int16 value. eg. 172
+length of first item is 172. 
+
+68.6.1 Table row layout - And item is further made up of a fixed-size header (23 bytes), followed by optional data, then is the actual user data.
+the pointer to the first value -> data inspector -> Int16 -> eg. 203 is the actual data is the id of the data in the postgreSQL table.