云野 New-运维运维 2023-05-18

构建nginx高速缓存

使用memc-nginx和srcache-nginx模块构建高效透明的缓存机制

传统缓存策略:

高效缓存策略:

OpenResty简介：

OpenResty^® 是一个基于 Nginx 与 Lua 的高性能 Web 平台，其内部集成了大量精良的 Lua 库、第三方模块以及大多数的依赖项。用于方便地搭建能够处理超高并发、扩展性极高的动态 Web 应用、Web 服务和动态网关。

OpenResty^® 通过汇聚各种设计精良的 Nginx 模块（主要由 OpenResty 团队自主开发），从而将 Nginx 有效地变成一个强大的通用 Web 应用平台。这样，Web 开发人员和系统工程师可以使用 Lua 脚本语言调动 Nginx 支持的各种 C 以及 Lua 模块，快速构造出足以胜任 10K 乃至 1000K 以上单机并发连接的高性能 Web 应用系统。

OpenResty^® 的目标是让你的Web服务直接跑在 Nginx 服务内部，充分利用 Nginx 的非阻塞 I/O 模型，不仅仅对 HTTP 客户端请求,甚至于对远程后端诸如 MySQL、PostgreSQL、Memcached 以及 Redis 等都进行一致的高性能响应。

以上来自OpenResty官网

官网下载地址: https://openresty.org/cn/

之所以用openresty，是因为nginx里面没有高效缓存策略所需要的模块，要在nginx官网去下载然后重新编译进去比较麻烦，而openresty是nginx另外的发行版本，已经内置了开发人员所需要的很多主流模块。修改配置文件后会动态生效，比较方便。但是我们之前已经安装nginx，所以要先停掉原来的nginx服务，使用nginx的命令也要使用绝对路径，不然会和OpenResty冲突。

[root@php ]# nginx -s stop
[root@php ]# netstat -antlp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name 
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN 425/php-fpm: master 
tcp 0 0 0.0.0.0:11211 0.0.0.0:* LISTEN 7586/memcached 
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 3325/sshd 
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 3421/master 
tcp 0 0 172.25.32.2:22 172.25.32.250:41046 ESTABLISHED 3586/sshd: root@pts 
tcp6 0 0 :::11211 :::* LISTEN 7586/memcached 
tcp6 0 0 :::22 :::* LISTEN 3325/sshd 
tcp6 0 0 ::1:25 :::* LISTEN 3421/master

传统缓存策略

1. 解压，编译安装OpenResty

[root@php ~]# tar zxf openresty-1.21.4.1.tar.gz 
[root@php ~]# cd openresty-1.21.4.1/
[root@php openresty-1.21.4.1]# ls
bundle configure COPYRIGHT patches README.markdown README-windows.txt util
[root@php openresty-1.21.4.1]# ./configure --with-http_ssl_module --with-http_stub_status_module --prefix=/usr/local/openresty
platform: linux (linux)
cp -rp bundle/ build
cd build
cd LuaJIT-2.1-20220411
gmake TARGET_STRIP=@: CCDEBUG=-g XCFLAGS='-DLUAJIT_ENABLE_LUA52COMPAT' CC=cc PREFIX=/usr/local/openresty/luajit
==== Building LuaJIT 2.1.0-beta3 ====
gmake -C src
gmake[1]: Entering directory `/root/openresty-1.21.4.1/build/LuaJIT-2.1-20220411/src'
CC lj_init.o
HOSTCC host/minilua.o
.......

[root@php openresty-1.21.4.1]# make && make install

2. 将原来的nginx配置文件复制到openresty中的nginx配置文件目录中,检测一下语法

[root@php openresty-1.21.4.1]# cd /usr/local/openresty/nginx/conf/
[root@php conf]# ls
fastcgi.conf fastcgi_params koi-utf mime.types nginx.conf scgi_params uwsgi_params win-utf
fastcgi.conf.default fastcgi_params.default koi-win mime.types.default nginx.conf.default scgi_params.default uwsgi_params.default
[root@php conf]# cp /usr/local/nginx/conf/nginx.conf .
cp: overwrite ‘./nginx.conf’? y
[root@php conf]# /usr/local/openresty/nginx/sbin/nginx -t
nginx: the configuration file /usr/local/openresty/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/openresty/nginx/conf/nginx.conf test is successful

3. 将memcache和php的测试页复制到openresty下的nginx的html默认发布目录

[root@php conf]# cd /usr/local/openresty/nginx/html/
[root@php html]# cp /usr/local/nginx/html/example.php .
[root@php html]# cp /usr/local/nginx/html/index.php .
[root@php html]# ls
50x.html example.php index.html index.php

现在传统的缓存策略就好了，可以压测一下，和后面的高效缓存策略进行对比。

[root@nginx ~]# ab -c5 -n 5000 http://172.25.32.2/example.php
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 172.25.32.2 (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software: nginx/1.22.0
Server Hostname: 172.25.32.2
Server Port: 80

Document Path: /example.php
Document Length: 228 bytes

Concurrency Level: 5
Time taken for tests: 0.475 seconds
Complete requests: 5000
Failed requests: 0
Write errors: 0
Total transferred: 1955000 bytes
HTML transferred: 1140000 bytes
Requests per second: 10522.44 [#/sec] (mean)
Time per request: 0.475 [ms] (mean)
Time per request: 0.095 [ms] (mean, across all concurrent requests)
Transfer rate: 4017.85 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 1
Processing: 0 0 0.1 0 1
Waiting: 0 0 0.1 0 1
Total: 0 0 0.1 0 1

Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 1
80% 1
90% 1
95% 1
98% 1
99% 1
100% 1 (longest request)

构建高效缓存策略

1 . 编辑从原来nginx复制到的openresty下nginx的配置文件,在配置文件中构建高效缓存，然后检测语法重载

[root@php html]# cd /usr/local/openresty/nginx/conf/
[root@php conf]# ls
fastcgi.conf fastcgi_params koi-utf mime.types nginx.conf scgi_params uwsgi_params win-utf
fastcgi.conf.default fastcgi_params.default koi-win mime.types.default nginx.conf.default scgi_params.default uwsgi_params.default
[root@php conf]# vim nginx.conf

17 http {
18 upstream memcache {
19 server 127.0.0.1:11211;
20 keepalive 512;     // 保持 512 个不立即关闭的连接用于提升性能
21 }

.......
67 location /memc {
68 internal;        //表示只接受内部访问
69 memc_connect_timeout 100ms;
70 memc_send_timeout 100ms;
71 memc_read_timeout 100ms;
72 set $memc_key $query_string;       //使用内置的$query_string来作为key
73 set $memc_exptime 300;               //表示缓存失效时间
74 memc_pass memcache;
75 }

......

 80 location ~ \.php$ {   ##当所请求的 uri 以“ .php ”结尾时,首先到 memcache 中查询有没有以 $uri$args 为 key 的数据,如果有则直接返回;否则,执行 location 的逻辑,如果返回的 http 状态码为 200 ,则在输出前以$uri$args 为 key ,将输入结果存入 memcache 
81 set $key $uri$args;
82 srcache_fetch GET /memc $key;
83 srcache_store PUT /memc $key;
84 root html;
85 fastcgi_pass 127.0.0.1:9000;
86 fastcgi_index index.php;
87 #fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
88 include fastcgi.conf;
89 }

[root@php conf]# /usr/local/openresty/nginx/sbin/nginx -t
nginx: the configuration file /usr/local/openresty/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/openresty/nginx/conf/nginx.conf test is successful
[root@php conf]# /usr/local/openresty/nginx/sbin/nginx
[root@php conf]# netstat -antlp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name 
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN 425/php-fpm: master 
tcp 0 0 0.0.0.0:11211 0.0.0.0:* LISTEN 7586/memcached 
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 18722/nginx: master 
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 3325/sshd 
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 3421/master 
tcp 0 0 172.25.32.2:22 172.25.32.250:41046 ESTABLISHED 3586/sshd: root@pts 
tcp6 0 0 :::11211 :::* LISTEN 7586/memcached 
tcp6 0 0 :::22 :::* LISTEN 3325/sshd 
tcp6 0 0 ::1:25

构建高效缓存后测试

[root@nginx ~]# ab -c5 -n 5000 http://172.25.32.2/example.php
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 172.25.32.2 (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software: openresty/1.21.4.1
Server Hostname: 172.25.32.2
Server Port: 80

Document Path: /example.php
Document Length: 248 bytes

Concurrency Level: 5
Time taken for tests: 0.301 seconds
Complete requests: 5000
Failed requests: 0
Write errors: 0
Total transferred: 2190000 bytes
HTML transferred: 1240000 bytes
Requests per second: 16628.75 [#/sec] (mean)
Time per request: 0.301 [ms] (mean)
Time per request: 0.060 [ms] (mean, across all concurrent requests)
Transfer rate: 7112.69 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 0 0 0.1 0 1
Waiting: 0 0 0.1 0 1
Total: 0 0 0.1 0 1

Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 0
95% 0
98% 0
99% 0
100% 1 (longest request)

可以看到从访问速度会比之前快了很多

nginx

云野 » 构建nginx高速缓存

云野普通

分享到：

1 评论

普通 James R. Johnson

2023年 5月 27日 at 下午4:52 回复

I’ve been exploring for a little bit for any high-quality articles or weblog posts on this sort of house .
Exploring in Yahoo I eventually stumbled upon this web site.
Studying this information So i am happy to show that I’ve a very just right uncanny feeling I found out exactly what I needed.
I so much indisputably will make certain to don?t
forget this web site and provides it a look regularly.

使用memc-nginx和srcache-nginx模块构建高效透明的缓存机制

传统缓存策略

现在传统的缓存策略就好了，可以压测一下，和后面的高效缓存策略进行对比。

构建高效缓存策略

构建高效缓存后测试

可以看到从访问速度会比之前快了很多

云野 普通

相关推荐

1 评论

发表回复 取消回复

云野普通

发表回复取消回复