调优"四剑客"的实战演练,福尔摩斯•K带你轻松优化性能

1 几个重要的库 本文为摘录,原文为: https://mp.weixin.qq.com/s?__biz=MjM5NjgwMDMxMg==&mid=2652164045&idx=1&sn=305c8d05c0b8b41827a9d4062c44a4a8&chksm=bd03dd9a8a74548c907bc3f1ffac82db0207e14bdd6c2398fa798187b9ea449371a9f0ec1f6f&mpshare=1&scene=1&srcid=1205yZcXcCHhOMmAZpuiFyFo&sharer_sharetime=1688349494576&sharer_shareid=a1eb8d286d4dcd42a0c884a984d12f54#rd 1 几个重要的库 KWR : KES 自动负载信息库(Automatic Workload Repertories),每小时自动采集性能指标...

A Learned Query Rewrite System using Monte Carlo Tree Search

1 INTRODUCTION 2 LearnedRewrite 2.1 树形结构 3 PRELIMINARIES 3.1 Query Rewrite Rules 3.2 Query Rewrite 4 TREE SEARCH FOR QUERY REWRITE 4.1 Overview of Policy Tree Search 本文为摘录,原文为: attachments/pdf/7/p46-li.pdf 查询重写使用启发式算法来实现,有两个限制 规则的应用顺序严重...

A Study of Database Performance Sensitivity to Experiment Settings

Adaptive Range Filters for Cold Data: Avoiding Trips to Siberia

1 ABSTRACT 2 INTRODUCTION 3 APPLICATION EXAMPLE 3.1 Project Siberia 本文为摘录,原文为: attachments/pdf/f/p1714-kossmann.pdf 对目前的架构来讲,和 粗糙索引 作用重叠。。 1 ABSTRACT ARF: Adaptive Range Filter, 自适应范围过滤器 ARF is for Range queries, while BloomFilter is for Point queries 2 INTRODUCTION 3 APPLICATION...

All Major C++17 Features You Should Know

1 Language Features 1.1 New auto rules for direct-list-initialization 1.2 static_assert with no message 1.3 typename in a template template parameter 1.4 Nested namespace definition 本文为摘录,原文为: https://www.cppstories.com/2017/01/cpp17features/ 1 Language Features 1.1 New auto rules for direct-list-initialization 1.2 static_assert with no message static_assert() 可以不用再写 message 了, 类似 C assert 1.3 typename in a template template...

ByteHTAP: ByteDance’s HTAP System with High Data Freshness and Strong Data Consistency

1 INTRODUCTION 2 RELATED WORK 3 SYSTEM ARCHITECTURE 3.1 ByteNDB Overview 3.2 System Overview 3.3 Metadata Service 3.4 OLAP Engine 4 SHARED STORAGE WITH HIGH DATA FRESHNESS 4.1 Delta Store 4.2 Base Store 4.3 High Data Freshness 5 LSN-BASED STRONG DATA CONSISTENCY 6 OLAP QUERY PERFORMANCE OPTIMIZATION 6.1 Delete Optimization for Scans 6.2 Computation Pushdown to Storage Engine 6.3 OLAP Query Engine Optimization 本文为摘录,原...

Cardinality Estimation in DBMS: A Comprehensive Benchmark Evaluation

1 ABSTRACT 2 INTRODUCTION 3 PRELIMINARIES AND BACKGROUND 4 EVALUATION PLAN 4.1 CardEst Algorithms 本文为摘录,原文为: attachments/pdf/2/p752-zhu.pdf 1 ABSTRACT we compare the effectiveness of CardEst methods in a real DBMS We establish a new benchmark for CardEst, which: contains a new complex realworld dataset STATS and a diverse query workload STATS-CEB. We integrate multiple most representative CardEst methods into PostgreSQL, and evaluate...

cgroup memo

1 Threads (Tasks (threads) versus processes) 1.1 cgroup v2 线程限制 1 Threads (Tasks (threads) versus processes) cgroup v1 可以单独操作进程中的线程, man cgroups : In cgroups v1, a distinction is drawn between processes and tasks. In this view, a process can con‐ sist of multiple tasks (more commonly called threads, from a user-space perspective,...

Chimp: Efficient Lossless Floating Point Compression for Time Series Databases

1 ABSTRACT 2 INTRODUCTION 3 PRELIMINARIES 3.1 Floating Point Time Series 3.2 IEEE 754 Double Precision Floating Point Format 3.3 Gorilla Compression 4 PROPERTIES OF REAL-WORLD TIME SERIES 4.1 Trailing Zeros 4.2 Leading Zeros 4.3 Revisiting Gorilla Compression 5 OVERVIEW 5.1 Our Chimp Algorithm 6 效果对比 6.1 Compression size result 6.2 Compression and decompression time 本文为摘录,原文为: attachments/pdf/1/p3058-liakos.pdf 1...

Columnar Formats for Schemaless LSM-based Document Stores

1 INTRODUCTION 2 BACKGROUND 2.1 Apache AsterixDB 本文为摘录,原文为: attachments/pdf/f/p2085-alkowaileet.pdf 这篇论文探讨了文档存储数据库系统的数据模型灵活性限制其在列式关系数据库中进行分析工作负载 的表现。作者提出...

Coping with the TCP TIME-WAIT state on busy Linux servers

1 TL;DR 2 About the TIME-WAIT state 2.1 TCP state diagram 2.2 Purpose 3 Problems 4 Other solutions 4.1 net.ipv4.tcp_tw_reuse 5 Another 6 总结 本文为摘录,原文为: https://vincent.bernat.ch/en/blog/2014-tcp-time-wait-state-linux#summary 1 TL;DR 不用启用 net.ipv4.tcp_tw_recycle , 该选项已在 Linux4.12 废弃 多数情况下, TIME-WAIT 状态的 socket 无害 2 About the TIME-WAIT state...

Data Structures in PG

1 Hash & TapeSet 2 Slot & tuple 1 Hash & TapeSet class LogicalTapeSet { + BufFile pfile + SharedFileSet fileset + int worker + long nBlocksAllocated + long nBlocksWritten + long nHoleBlocks + bool forgetFreeSpace + long freeBlocks + long nFreeBlocks + Size freeBlocksLen + bool enable_prealloc } class LogicalTape { + LogicalTapeSet tapeSet + bool writing + bool frozen + bool dirty + long firstBlockNumber + long curBlockNumber + long nextBlockNumber + long offsetBlockNumber + char buffer + int buffer_size + int max_size + int pos + int nbytes + long prealloc + int nprealloc + int prealloc_size } class BufFile { + int numFiles + File files + bool isInterXact + bool dirty + bool readOnly + FileSet fileset + const name + ResourceOwner resowner + int curFile + off_t curOffset + int pos + int nbytes + PGAlignedBlock buffer } LogicalTapeSet *-- BufFile class HashAggSpill { + int npartitions + LogicalTape partitions + int64 ntuples + uint32 mask + int shift + hyperLogLogState hll_card } HashAggSpill *-- LogicalTape LogicalTape - LogicalTapeSet 2 Slot & tuple class TupleTableSlot { + NodeTag type + int tts_flags + AttrNumber tts_nvalid + const tts_ops + TupleDesc tts_tupleDescriptor + int tts_values + int tts_isnull + int tts_mcxt + ItemPointerData tts_tid + int tts_tableOid } class MinimalTupleData { + int t_len + char mt_padding + int t_infomask2 + int t_infomask + int t_hoff + int t_bits } class HeapTupleHeaderData { + union t_choice + ItemPointerData t_ctid + int t_infomask2 + int t_infomask + int t_hoff + int t_bits } class union { + HeapTupleFields t_heap + DatumTupleFields t_datum } HeapTupleHeaderData *-- union class MinimalTupleTableSlot { + TupleTableSlot base + HeapTuple tuple + MinimalTuple mintuple + HeapTupleData minhdr + int off } class HeapTupleData { + int t_len + ItemPointerData t_self + int t_tableOid + HeapTupleHeader t_data } class VirtualTupleTableSlot { + TupleTableSlot base + char data } class HeapTupleTableSlot { + TupleTableSlot base + HeapTuple tuple + int off + HeapTupleData tupdata } class BufferHeapTupleTableSlot { + HeapTupleTableSlot base + Buffer buffer } TupleTableSlot <|-- MinimalTupleTableSlot TupleTableSlot <|-- VirtualTupleTableSlot TupleTableSlot <|-- HeapTupleTableSlot HeapTupleTableSlot <|-- BufferHeapTupleTableSlot MinimalTupleTableSlot *-- MinimalTupleData HeapTupleTableSlot *-- HeapTupleData HeapTupleData *-- HeapTupleHeaderData

Design Trade-offs for a Robust Dynamic Hybrid Hash Join

1 ABSTRACT 2 INTRODUCTION 3 BACKGROUND 3.1 Hybrid Hash Join 本文为摘录,原文为: attachments/pdf/f/p2257-jahangiri.pdf 1 ABSTRACT 探索分片数量对 HHJ 性能的影响 propose a new lower bound for number of partitions design and evaluate different partition insertion techniques to maximize memory utilization with the lest CPU cost 2 INTRODUCTION HHJ: Hybrid Hash Join 算法 将...

GitHub - powa-team/powa: PostgreSQL Workload Analyzer

本文为摘录,原文为: https://github.com/powa-team/powa Workload Analyzer.

google chrome in hyprland

1 google chrome in hyprland 2 Weird crash when running chrome with: –gtk-version=4 1 google chrome in hyprland Hyprland is cool, but it may not work smoothly on default Google Chrome. The borders appear distorted and blurry. To resolve this issue, simply open the Chrome settings, navigate to appearance (chrome://settings/appearance), and enable “Use system title bar and borders.” Afterwards, restart the browser. 2 Weird crash when running chrome with: –gtk-version=4 This happens when use GTK theme as shown in above screenshot....

gp shared snapshot

1 DtaStrutor 本文为摘录,原文为: ../../../Work/pg_gpdb/src/backend/utils/time/sharedsnapshot.c 在 Greenplum 中,作为切片计划的一部分,许多 PostgreSQL 进程(qExecs,QE)在单个段数据库 上运行,作为同一用户 SQL 语句的一部分。属...

gpcheckcat

1 概述 2 pg_class 3 namespace 4 unique_index_violation 5 duplicate 1 概述 GP 提供了 gpcheckcat 用于在集群内检查系统表。 | 检查项 || 描述 || Utility 模式 || 复合查询 || 错误等级 | |———————|————————————————————————-|————|——|——| | pg_class || Check pg_class entry that does not have any correspond pg_attribute entry || Y...

gpdb cdb

1 Data structures 1.1 Slice Table 2 PostgresMain 2.1 Call graph (QE): 2.2 QD 3 Receiver 4 Sender 1 Data structures 1.1 Slice Table @startuml class SliceTable { + NodeTag type + int localSlice + int numSlices + ExecSlice slices + bool hasMotions + int instrument_options + uint32 ic_instance_id } note right of SliceTable::localSlice Index of the slice to execute end note note right of SliceTable::slices Array of slices, indexed...

gpdb memory control

1 TODO VMem 2 Resource Group Control 2.1 Bypass 2.2 Fixed Mem 2.3 Memory Limit TODO 1 VMem 2 Resource Group Control ResourceGroupGetQueryMemoryLimit(void) 用于获取内存限制的绝对大小 (非百分比) 2.1 Bypass 绕开资源限制模式 Enabled when: gp_resource_group_bypass is true: guc_gp.c { {"gp_resource_group_bypass", PGC_USERSET, RESOURCES, gettext_noop("If the value is true, the query in...

GPDB: Configuring Your Systems

1 IP Fragmentation Settings 2 Networking 本文为摘录,原文为: https://docs.vmware.com/en/VMware-Tanzu-Greenplum/6/greenplum-database/GUID-install_guide-prep_os.html 1 IP Fragmentation Settings IP Fragmentation Settings When the Greenplum Database interconnect uses UDP (the default), the network interface card controls IP packet fragmentation and reassemblies. If the UDP message size is larger than the size of the maximum transmission unit (MTU) of a network, the IP layer fragments the message. (Refer to...