分布式下Session一致性架構(gòu)舉例

一、問(wèn)題及方案

見(jiàn)這篇文章:分布式下Session一致性問(wèn)題

二、分布式環(huán)境搭建:

image

系統(tǒng)環(huán)境

[root@centos7 ~]# cat /etc/redhat-release 
CentOS Linux release 7.4.1708 (Core) 
[root@centos7 ~]# 

2.1 安裝jdk

# 下載jdk-8u141-linux-x64.tar.gz
# 創(chuàng)建目錄
mkdir -p /opt/java
# 解壓
tar -xzvf jdk-8u141-linux-x64.tar.gz -C /opt/java
# 創(chuàng)建鏈接
ln -s /opt/java/jdk1.8.0_141 /usr/local/jdk
# 設(shè)置變量 /etc/profile末尾添加
export JAVA_HOME=/usr/local/jdk
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
# 使變量生效
source /etc/profile
export PATH=$JAVA_HOME/bin:$PATH
# 驗(yàn)證
java -version

2.2 安裝tomcat

安裝3個(gè)tomcat并設(shè)置其端口號(hào)

# 下載apache-tomcat-8.5.16.tar.gz
# 創(chuàng)建目錄
mkdir -p /opt/tomcat
# 解壓
tar -xzvf apache-tomcat-8.5.16.tar.gz -C /opt/tomcat
# 清空ROOT目錄
rm -rf /opt/tomcat/apache-tomcat-8.5.16/webapps/ROOT/*
# 復(fù)制3個(gè)tomcat
cd /opt/tomcat
cp -a apache-tomcat-8.5.16 tomcat1
cp -a apache-tomcat-8.5.16 tomcat2
cp -a apache-tomcat-8.5.16 tomcat3
# 修改端口號(hào)
sed -i '22s/8005/8'"$i"'05/' $file
sed -i '69s/8080/8'"$i"'80/' $file
sed -i '116s/8009/8'"$i"'09/' $file

修改如下3個(gè)端口號(hào)

image

用如下腳本修改:a.sh

#!/bin/sh

for i in {1..3}
do
  file=/opt/tomcat/tomcat"$i"/conf/server.xml
  sed -i '22s/8005/8'"$i"'05/' $file
  sed -i '69s/8080/8'"$i"'80/' $file
  sed -i '116s/8009/8'"$i"'09/' $file
  #sed -i '148s#appBase=".*"#appBase="/data/webapps"#' $file
done

執(zhí)行腳本修改端口

sh a.sh
執(zhí)行后,3個(gè)tomcat的端口號(hào)分別改為
8180,8280,8380

啟動(dòng)并驗(yàn)證這3個(gè)tomcat

# 分別向3個(gè)tomcat寫(xiě)入一個(gè)測(cè)試文件a.txt
for i in {1..3};do echo 8"$i"80>/opt/tomcat/tomcat"$i"/webapps/ROOT/a.txt;done

# 啟動(dòng)
for i in {1..3};do /opt/tomcat/tomcat"$i"/bin/startup.sh;done

# 驗(yàn)證
for i in {1..3};do curl 127.0.0.1:8"$i"80/a.txt;done
image

2.3 安裝nginx

# 下載nginx-1.13.9.tar.gz
mkdir -p /opt/nginx/nginx-1.3.13.9
# 創(chuàng)建用戶
useradd nginx -s /sbin/nologin -M
# 安裝pcre openssl
yum install pcre pcre-devel openssl openssl-devel -y
# 解壓、編譯安裝
tar -xzvf nginx-1.13.9.tar.gz
cd nginx-1.13.9
./configure --user=nginx --group=nginx --prefix=/opt/nginx/nginx-1.13.9 --with-http_stub_status_module --with-http_ssl_module
make
make install
# 創(chuàng)建鏈接
ln -s /opt/nginx/nginx-1.13.9 /usr/local/nginx
# 設(shè)置環(huán)境變量
echo 'export PATH=/usr/local/nginx/sbin:$PATH' >> /etc/profile
source /etc/profile

nginx常用命令

# 啟動(dòng)
nginx
# 檢查配置文件
nginx -t
# 平滑啟動(dòng)
nginx -s reload
# 檢查
netstat -tunlp | grep nginx

遇到的問(wèn)題:

nginx: [emerg] getpwnam("nginx") failed
未創(chuàng)建nginx用戶

配置nginx

/usr/local/nginx/conf/nginx.conf

worker_processes  1;
events {
    worker_connections  1024;
}

# http最外層模塊
http {
    # 全局變量參數(shù)
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    upstream web_pool {
        server 127.0.0.1:8180 weight=1;
        server 127.0.0.1:8280 weight=1;
        server 127.0.0.1:8380 weight=2;
    }

    # server相當(dāng)于虛擬站點(diǎn)
    server {
        listen       80;
        server_name  localhost;
        location / {
            proxy_pass  http://web_pool;
            index  index.html index.htm;
        }
    }
}

測(cè)試一下

for i in {1..9};do curl 127.0.0.1/a.txt;done
image

2.4 安裝redis

wget http://download.redis.io/releases/redis-4.0.8.tar.gz
mkdir -p /opt/redis
tar -xzvf redis-4.0.8.tar.gz -C /opt/redis/
ln -s /opt/redis/redis-4.0.8 /usr/local/redis
cd /usr/local/redis
make && make install

# 配置讓其他機(jī)器也可以訪問(wèn)
vi /usr/local/redis/redis.conf
#bind 127.0.0.1
bind 0.0.0.0

操作redis

# 查看redis版本
redis-cli -v
# 啟動(dòng)redis
redis-server redis.conf > /tmp/redis.log 2>&1 &
# or
nohup redis-server redis.conf &
# 監(jiān)控redis
redis-cli -h 192.168.234.130 -p 6379 monitor
# 關(guān)閉redis
redis-cli -h 127.0.0.1 -p 6379 shutdown

客戶端

# 連接
redis-cli -h 192.168.5.220
# 查看
keys *
# 清空
flushall

三、問(wèn)題重現(xiàn)

3.1 web應(yīng)用

java-web-login

只實(shí)現(xiàn)登錄功能

/login   登錄頁(yè)面,已登錄跳轉(zhuǎn)到/index
/logout  退出
/index   首頁(yè),未登錄跳轉(zhuǎn)到/login

將這個(gè)應(yīng)用部署到3個(gè)tomcat上

重啟tomcat

for i in {1..3};do /opt/tomcat/tomcat"$i"/bin/shutdown.sh;done
for i in {1..3};do /opt/tomcat/tomcat"$i"/bin/startup.sh;done
image
image

3.2 問(wèn)題描述

登錄界面是8280端口,輸入用戶名密碼點(diǎn)擊登錄,由于nginx配置的是基于輪詢的算法進(jìn)行分發(fā),8380端口的服務(wù)器處理的登錄,因此session創(chuàng)建于8380端口,而8280處于非登錄狀態(tài)。

四、解決實(shí)現(xiàn)

4.1 session sticky

修改nginx的分發(fā)算法改為基于ip

upstream web_pool {
    # 默認(rèn)是輪詢
    ip_hash;
    server 127.0.0.1:8180 weight=1;
    server 127.0.0.1:8280 weight=1;
    server 127.0.0.1:8380 weight=2;
}

修改后重啟nginx

nginx -t
nginx -s reload

4.2 session replication

http://tomcat.apache.org/tomcat-8.5-doc/cluster-howto.html

tomcat配置server.xml

<!--修改每個(gè)tomcat下的conf/server.xml-->
<!--其中Receiver這個(gè)結(jié)點(diǎn)的端口分別配置為4001,4002,4003-->
<!--channelSendOptions-->
<!--8表示異步發(fā)送 2表示確認(rèn)發(fā)送 4表示同步發(fā)送 10表示同步+確認(rèn)-->
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" 
    channelSendOptions="8">
    <Manager className="org.apache.catalina.ha.session.DeltaManager" 
        expireSessionsOnShutdown="false" 
        notifyListenersOnReplication="true" />
    <Channel className="org.apache.catalina.tribes.group.GroupChannel">
        <!-- 228.0.0.4是主播地址-->
        <Membership className="org.apache.catalina.tribes.membership.McastService"
            address="228.0.0.4" 
            port="45564" 
            frequency="500" 
            dropTime="3000" />
        <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
            address="auto" 
            port="4001" 
            autoBind="100" 
            selectorTimeout="5000"
            maxThreads="6" />
        <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
            <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" />
        </Sender>
        <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector" />
        <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor" />
    </Channel>
    <Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter="" />
    <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve" />
    <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
        tempDir="/tmp/war-temp/" 
        deployDir="/tmp/war-deploy/" 
        watchDir="/tmp/war-listen/"
        watchEnabled="false" />
    <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener" />
</Cluster>

tomcat的配置web.xml

# 修改每個(gè)tomcat下的conf/web.xml不起作用
# 一定要修改每個(gè)應(yīng)用的web.xml,在<web-app>這個(gè)節(jié)點(diǎn)下添加
<distributable />

4.3 session數(shù)據(jù)集中存儲(chǔ)

數(shù)據(jù)存儲(chǔ)到redis,使用spring-data-redis

見(jiàn)應(yīng)用java-web-login

把web_spring-redis.xml重命名為web.xml,部署到tomcat即可,得先啟動(dòng)redis參照“安裝redis”這一節(jié)。

spring-redis.xml

    <?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
       http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">

    <!-- 引入配置文件 -->
    <bean id="propertyConfigurer"
        class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="locations">
            <list>
                <value>classpath*:redis.properties</value>
            </list>
        </property>
    </bean>

    <bean id="redisHttpSessionConfiguration"
        class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration">
        <property name="maxInactiveIntervalInSeconds" value="600"/>
    </bean>

    <bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig">
        <property name="maxTotal" value="${redis.pool.maxTotal}" />
        <property name="maxIdle" value="${redis.pool.maxIdle}" />
    </bean>

    <bean id="jedisConnectionFactory"
        class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"
        destroy-method="destroy">
        <property name="hostName" value="${redis.hostname}" />
        <property name="port" value="${redis.port}" />
        <property name="timeout" value="${redis.timeout}" />
        <property name="usePool" value="true" />
        <property name="poolConfig" ref="jedisPoolConfig" />
    </bean>
</beans>

注意:

  • Spring版本的選擇 >=4.0.3

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.yysue</groupId>
    <artifactId>java-web-login</artifactId>
    <packaging>war</packaging>
    <name>java-web-login</name>
    <version>0.0.1-SNAPSHOT</version>

    <properties>
        <java.version>1.8</java.version>
        <spring.version>4.0.3.RELEASE</spring.version>
    </properties>

    <!-- 依賴 -->
    <dependencies>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
            <version>3.0.1</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>javax.servlet.jsp</groupId>
            <artifactId>jsp-api</artifactId>
            <version>2.2</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>jstl</artifactId>
            <version>1.2</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-oxm</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-tx</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context-support</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-aop</artifactId>
            <version>${spring.version}</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-redis</artifactId>
            <version>1.5.2.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.session</groupId>
            <artifactId>spring-session-data-redis</artifactId>
            <version>1.2.2.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <version>2.8.1</version>
        </dependency>

    </dependencies>

    <build>
        <!-- 插件 -->
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書(shū)系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容