we compare the effectiveness of CardEst methods in a real DBMS
We establish a new benchmark for CardEst, which:
- contains a new complex realworld dataset STATS and
- a diverse query workload STATS-CEB.
We integrate multiple most representative CardEst methods into PostgreSQL,

cgroup memo

yc 发布于 2022-05-01

Table of Contents

1 Threads (Tasks (threads) versus processes)
- 1.1 cgroup v2 线程限制

1 Threads (Tasks (threads) versus processes)

cgroup v1 可以单独操作进程中的线程, man cgroups :

In cgroups v1, a distinction is drawn between processes and tasks. In this view, a process can con‐ sist of multiple tasks (more commonly called threads, from a user-space perspective, and called such in the remainder of this man page). In cgroups v1, it is possible to independently manipulate the cgroup memberships of the threads in a process.

Columnar Formats for Schemaless LSM-based Document Stores

yc 发布于 2022-05-01

Table of Contents

1 INTRODUCTION
2 BACKGROUND
- 2.1 Apache AsterixDB

本文为摘录(或转载)，侵删，原文为： attachments/pdf/f/p2085-alkowaileet.pdf

这篇论文探讨了文档存储数据库系统的数据模型灵活性限制其在列式关系数据库中进行分析工作负载的表现。作者提出了几种基于 Log-Structured Merge (LSM) tree 事件的技术，旨在将数据存储在列式布局中。他们扩展了 Dremel 格式并引入一个新的列式布局来组织和存储数据。论文还强调了在文档存储中使用查询编译技术的潜力，并介绍了他们在 Apache AsterixDB 中的实现和评估。实验结果显示了显著的性能增益，并在最小程度上影响了摄入性能。

Data Structures in PG

yc 发布于 2022-05-01

Table of Contents

1 Hash & TapeSet
2 Slot & tuple

1 Hash & TapeSet

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

class LogicalTapeSet {
    + BufFile pfile
    + SharedFileSet fileset
    + int worker
    + long nBlocksAllocated
    + long nBlocksWritten
    + long nHoleBlocks
    + bool forgetFreeSpace
    + long freeBlocks
    + long nFreeBlocks
    + Size freeBlocksLen
    + bool enable_prealloc
}

class LogicalTape {
    + LogicalTapeSet tapeSet
    + bool writing
    + bool frozen
    + bool dirty
    + long firstBlockNumber
    + long curBlockNumber
    + long nextBlockNumber
    + long offsetBlockNumber
    + char buffer
    + int buffer_size
    + int max_size
    + int pos
    + int nbytes
    + long prealloc
    + int nprealloc
    + int prealloc_size
}



class BufFile {
    + int numFiles
    + File files
    + bool isInterXact
    + bool dirty
    + bool readOnly
    + FileSet fileset
    + const name
    + ResourceOwner resowner
    + int curFile
    + off_t curOffset
    + int pos
    + int nbytes
    + PGAlignedBlock buffer
}

LogicalTapeSet *-- BufFile

class HashAggSpill {
+ int npartitions
+ LogicalTape partitions
+ int64 ntuples
+ uint32 mask
+ int shift
+ hyperLogLogState hll_card
}


HashAggSpill *-- LogicalTape

LogicalTape - LogicalTapeSet

2 Slot & tuple

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

class TupleTableSlot {
  + NodeTag type
  + int tts_flags
  + AttrNumber tts_nvalid
  + const tts_ops
  + TupleDesc tts_tupleDescriptor
  + int tts_values
  + int tts_isnull
  + int tts_mcxt
  + ItemPointerData tts_tid
  + int tts_tableOid
}




class MinimalTupleData {
  + int t_len
  + char mt_padding
  + int t_infomask2
  + int t_infomask
  + int t_hoff
  + int t_bits
}


class HeapTupleHeaderData {
  + union t_choice
  + ItemPointerData t_ctid
  + int t_infomask2
  + int t_infomask
  + int t_hoff
  + int t_bits
}

class union {
  + HeapTupleFields t_heap
  + DatumTupleFields t_datum
}


HeapTupleHeaderData *-- union

class MinimalTupleTableSlot {
  + TupleTableSlot base
  + HeapTuple tuple
  + MinimalTuple mintuple
  + HeapTupleData minhdr
  + int off
}

class HeapTupleData {
  + int t_len
  + ItemPointerData t_self
  + int t_tableOid
  + HeapTupleHeader t_data
}


class VirtualTupleTableSlot {
  + TupleTableSlot base
  + char data
}

class HeapTupleTableSlot {
  + TupleTableSlot base
  + HeapTuple tuple
  + int off
  + HeapTupleData tupdata
}

class BufferHeapTupleTableSlot {
  + HeapTupleTableSlot base
  + Buffer buffer
}


TupleTableSlot <|-- MinimalTupleTableSlot
TupleTableSlot <|-- VirtualTupleTableSlot
TupleTableSlot <|-- HeapTupleTableSlot
HeapTupleTableSlot <|-- BufferHeapTupleTableSlot


MinimalTupleTableSlot *-- MinimalTupleData
HeapTupleTableSlot *-- HeapTupleData
HeapTupleData *-- HeapTupleHeaderData

GitHub - powa-team/powa: PostgreSQL Workload Analyzer

yc 发布于 2022-05-01

本文为摘录(或转载)，侵删，原文为： https://github.com/powa-team/powa

Workload Analyzer.

gp shared snapshot

yc 发布于 2022-05-01

Table of Contents

1 DtaStrutor

本文为摘录(或转载)，侵删，原文为： ../../../Work/pg_gpdb/src/backend/utils/time/sharedsnapshot.c

在 Greenplum 中，作为切片计划的一部分，许多 PostgreSQL 进程（qExecs，QE）在单个段数据库上运行，作为同一用户 SQL 语句的一部分。属于特定用户在特定段数据库上的所有 qExecs 需要具有一致的可见性。为此，使用了一种称为“共享本地快照”（Shared Local Snapshot）的思想。共享内存数据结构 SharedSnapshotSlot 在特定数据库实例上的会话的统一流程（gang processes）之间共享会话和事务信息。这些流程被称为 SegMate 进程组。