分类目录归档:性能和可伸缩性

Performance and Scalability

Linux High Availability RHCS

一、简介

RHCS 即 RedHat Cluster Suite ,中文意思即红帽集群套件。红帽集群套件(RedHat Cluter Suite, RHCS)是一套综合的软件组件,可以通过在部署时采用不同的配置,以满足你的对高可用性,负载均衡,可扩展性,文件共享和节约成本的需要。对于需要最大 正常运行时间的应用来说,带有红帽集群套件(Red Hat Cluster Suite)的红帽企业 Linux 集群是最佳的选择。红帽集群套件专为红帽企业 Linux 量身设计,它提供有如下两种不同类型的集群: 1、应用/服务故障切换-通过创建n个节点的服务器集群来实现关键应用和服务的故障切换 2、IP 负载均衡-对一群服务器上收到的 IP 网络请求进行负载均衡利用红帽集群套件,可以以高可用性配置来部署应用,从而使其总是处于运行状态-这赋予了企业向外扩展(scale- out)Linux 部署的能力。对于网络文件系统(NFS)、Samba 和Apache 等大量应用的开源应用来说,红帽集群套件提供了一个随时可用的全面故障切换解决方案。 继续阅读

Linux Cluster

This is the first part of my “Linux Cluster” posts:

  • Linux Cluster Part 1 – Install Corosync and Pacemaker on CentOS 6 – Learn how to install Corosync and Pacemaker on CentOS 6
  • Linux Cluster Part 2 – Adding and Deleting Cluster Resources – Learn how to add and delete Linux Cluster Resources and how to use CRM Shell
  • Linux Cluster Part 3 – Manage Cluster Nodes and Resources – Learn how to manage Linux Cluster Nodes (maintenance mode, standby mode, …) and Linux Cluster Resources (resource constraints – order, colocation, …)

Linux Cluster Part 1 – Install Corosync and Pacemaker on CentOS 6

I have been using Linux Cluster Engine called Corosync and Linux Cluster Resource Manager called Pacemaker for a while now and must say i am very satisfied with it. Corosync and Pacemaker combined can turn your Linux boxes into a Linux High Availability Cluster.

Corosync and Pacemaker Linux Cluster of course supports both Active/Passive and Active/Active modes on multi-nodes!
pcmk-active-passive-small-300x225

Linux Cluster (source: clusterlabs.org)

继续阅读

504 Gateway Time-out using Nginx

It is very common to see a 504 Gateway Time-out using Nginx webserver. This timeout error is generated often by a number of reasons on the backend connection that is serving content. To fix it, you will have to figure out what configuration are you using. 继续阅读

Proxmox VE 3.3 2-node cluster with GlusterFS

This article covers setup of simple and cost-effective 2-node Proxmox VE cluster featuring locally installed GlusterFS as shared filesystem for cluster VMs. Through this solution is not intended for mission critical or enterprise needs, its ease and ability to run on bargain priced hardware makes it interesting for non-profit organisations, labs or clustering enthusiasts.

gluster

 


This HOWTO implies you have 2 freshly installed Proxmox VE 3.3 nodes:
pve-node-01 (IPv4 10.10.0.1) and pve-node-02 (IPv4 10.10.0.2) connected to the 10.10.0.0/16 private network. Keep in mind that your actual setup may differ and you may need to change some of commands in this HOWTO to suit your needs. 继续阅读

tcpcopy

TCPCopy – A TCP Stream Replay Tool

TCPCopy is a TCP stream replay tool to support real testing of Internet server applications.

Description

Although the real live flow is important for the test of Internet server applications, it is hard to simulate it as online environments are too complex. To support more realistic testing of Internet server applications, we develop a live flow reproduction tool – TCPCopy, which could generate the test workload that is similar to the production workload. Currently, TCPCopy has been widely used by companies in China.

TCPCopy has little influence on the production system except occupying additional CPU, memory and bandwidth. Moreover, the reproduced workload is similar to the production workload in request diversity, network latency and resource occupation. 继续阅读

Setting up Tomcat Cluster for Session Replication

If you have your web application running on one tomcat instance and want to add another tomcat instance (ideally on a different machine), following steps will guide you.

Step 1: Independently deploy your web application (WAR file) on each instance and make sure they can work independently.

Step 2: Stop tomcat

Step 3: Update the <Cluster> element under the <Engine> element in the Server.xml file (under the conf dir in tomcat installation dir) on both your servers with:

<Engine name="<meaningful_unique_name>" defaultHost="localhost">      
     <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
              channelSendOptions="8">
          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>
          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
               <Membership className="org.apache.catalina.tribes.membership.McastService"
                           address="228.0.0.4"
                           port="45564"
                           frequency="500"
                           dropTime="3000"/>
               <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                         address="auto"
                         port="4000"
                         autoBind="100"
                         selectorTimeout="5000"
                         maxThreads="6"/>
               <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                   <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
               </Sender>
               <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
               <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
          </Channel>
          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.css;.*\.txt;"/>
          <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
     </Cluster>
     ...
</Engine>

For more details on these parameters, check https://sec1.woopra.com/docs/cluster-howto.html 继续阅读

HAProxy’s load-balancing algorithm for static content delivery with Varnish

HAProxy’s load-balancing algorithms

HAProxy supports many load-balancing algorithms which may be used in many different type of cases.
That said, cache servers, which deliver most of the time the static content from your web applications, may require some specific load-balancing algorithms.

HAProxy stands in front of your cache server for some good reasons:

  • SSL offloading (read PHK’s feeling about SSL, Varnish and HAProxy)
  • HTTP content switching capabilities
  • advanced load-balancing algorithms

The main purpose of this article is to show how HAProxy can be used to aggregate Varnish servers memory storage in some kind of “JBOD” mode (like the “Just a Bunch Of Disks“).
Main purpose of the examples delivered here are to optimize the resources on the cache, mainly its memory, in order to improve the HIT rate. This will also improve your application response time and make your site top ranked on google :) 继续阅读

Use a load-balancer as a first row of defense against DDOS

We’ve seen recently more and more DOS and DDOS attacks. Some of them were very big, requiring thousands of computers…
But in most cases, this kind of attacks are made by a few computers aiming to make a service or website unavailable, either by sending it too many requests or by taking all its available resources, preventing regular users to use the service.
Some attacks targets known vulnerabilities of widely used applications.

In the present article, we’ll explain how to take advantage of an application delivery controller to protect your website and application against DOS, DDOS and vulnerability scans.

Why using a LB for such protection since a firewall and a Web Application Firewall (aka WAF) could already do the job?
Well, the Firewall is not aware of the application layer but would be useful to pretect against SYN flood attacks. That’s why we saw recently application layer firewalls: Web Application Firewalls, also known as WAF.
Well, since the load balancer is in front of the platform, it can be a good partner for the WAF, filtering out 99% of the attacks, which are managed by script kiddies. The WAF can then happily clean up the remaining attacks.
Well, maybe you don’t need a WAF and you want to take advantage of your Aloha and save some money ;).

Note that you need an application layer load-balancer, like Aloha or OpenSource HAProxy to be efficient. 继续阅读

load balancing, affinity, persistence, sticky sessions: what you need to know

Synopsis

To ensure high availability and performance of Web applications, it is now common to use a load-balancer.
While some people uses layer 4 load-balancers, it can be sometime recommended to use layer 7 load-balancers to be more efficient with HTTP protocol.

NOTE: To understand better the difference between such load-balancers, please read the Load-Balancing FAQ.

A load-balancer in an infrastructure

The picture below shows how we usually install a load-balancer in an infrastructure:

This is a logical diagram. When working at layer 7 (aka Application layer), the load-balancer acts as a reverse proxy.
So, from a physical point of view, it can be plugged anywhere in the architecture:

  • in a DMZ
  • in the server LAN
  • as front of the servers, acting as the default gateway
  • far away in an other separated datacenter

继续阅读

How to run Docker containers on CentOS or Fedora

Lately Docker has emerged as a key technology for deploying applications in the cloud environment. Compared to traditional hardware virtualization, Docker-based container sandbox provides a number of advantages for application deployment environment, such as lightweight isolation, deployment portability, ease of maintenance, etc. Now Red Hat is steering community efforts in streamlining the management and deployment of Docker containers. 继续阅读