作者归档:flyinweb

Linux 磁盘IO性能测试工具:FIO

1、FIO安装

CentOS下:

$ yum install fio

2、FIO简介

FIO是测试IOPS的非常好的工具,用来对硬件进行压力测试和验证。
项目主页:https://github.com/axboe/fio
FIO输出结果详细解释可参见:https://tobert.github.io/post/2014-04-17-fio-output-explained.html
注:fio用于测试磁盘性能,不是文件系统,测试之前需要先把要测试的磁盘卸载掉,测试完后需格式化一遍再挂载上去。相关命令如下:

# 卸载磁盘
umount /mnt/datadir1
# 格式化磁盘
mkfs.ext4 /dev/sdd
# 挂载磁盘
mount -t ext4 /dev/sdd /mnt/datadir1

# 自动挂载,在里面添加该盘
vim /etc/fstab

3、FIO参数

例如:

$ fio -filename=/dev/sda -direct=1 -iodepth 1 -thread -rw=read -ioengine=psync -bs=16k -size=200G -numjobs=30 -runtime=1000 -group_reporting -name=mytest1

说明:
filename=/dev/sdd 测试文件名称,通常选择需要测试的盘的data目录。
direct=1 测试过程绕过机器自带的buffer。使测试结果更真实。
rw=randwrite 测试模式
=read 顺序读
=write 顺序写
=randwrite 随机写
=randread 随机读
=rw,readwrite 顺序混合读写
=randrw 随机混合读写

bs=4k 单次io的块文件大小为4k
bsrange=512-2048 同上,提定数据块的大小范围
size=5G 本次的测试文件大小为5g,以每次4k的io进行测试。
numjobs=30 本次的测试线程为30个
runtime=1000 测试时间为1000秒,如果不写则一直将5g文件分4k每次写完为止
iodepth 队列深度,只有使用libaio时才有意义。这是一个可以影响IOPS的参数。
ioengine=psync io引擎使用psync方式
=libaio Linux专有的异步IO

rwmixwrite=30 在混合读写的模式下,写占30%
group_reporting 关于显示结果的,汇总每个进程的信息。
lockmem=1G 只使用1g内存进行测试。
zero_buffers 用0初始化系统buffer。
nrfiles=8 每个进程生成文件的数量。

点击查看更多详细参数说明

4、FIO测试示例

fio工具使用特别注意:

  1. 请不要在系统盘上进行 fio 测试,避免损坏系统重要文件
  2. fio测试建议在空闲的、未保存重要数据的硬盘上进行,并在测试完后重新制作文件系统。请不要在业务数据硬盘上测试,避免底层文件系统元数据损坏导致数据损坏
  3. 测试硬盘性能时,推荐直接测试裸盘(如 vdb);测试文件系统性能时,推荐指定具体文件测试(如 /data/file)

4.1 顺序读测试

$ fio -ioengine=libaio -bs=4k -direct=1 -thread -rw=read -filename=/dev/sdd -name=”BS 4KB read test” -iodepth=16 -runtime=601

4.2 顺序写测试

$ fio -ioengine=libaio -bs=4k -direct=1 -thread -rw=write -filename=/dev/sdd -name=”BS 4KB write test” -iodepth=16 -runtime=601

4.3 随机读测试

$ fio -ioengine=libaio -bs=4k -direct=1 -thread -rw=randread -filename=/dev/sdd -name=”BS 4KB randread test” -iodepth=16 -runtime=601

4.4 随机写测试

$ fio -ioengine=libaio -bs=4k -direct=1 -thread -rw=randwrite -filename=/dev/sdd -name=”BS 4KB randwrite test” -iodepth=16 -runtime=601

4.5 随机读写混合测试

$ fio -ioengine=libaio -bs=4k -direct=1 -thread -rw=randrw -rwmixread=70 -filename=/dev/sdd -name=”BS 4KB randrw 70 test” -iodepth=16 -runtime=601

5、FIO输出结果

例如:

BS 4KB randread test: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.1.10
Starting 1 thread

BS 4KB randread test: (groupid=0, jobs=1): err= 0: pid=26408: Mon Feb 27 16:43:07 2017
read : io=19908KB, bw=339689B/s, iops=82, runt= 60013msec
# fio做了19908KB的IO,bandwidth速率为339689B/s,总IOPS为82,运行时间为60013毫秒

slat (usec): min=18, max=97, avg=23.60, stdev= 2.79
# slat (submission latency):代表 盘需要多久将IO提交到kernel做处理
# usec 微秒

clat (msec): min=1, max=23, avg=12.03, stdev= 3.79
# clat (completion latency):命令提交到kernel到IO做完之间的时间,不包括submission latency

 lat (msec): min=1, max=23, avg=12.05, stdev= 3.79
# 从IO结构体创建时刻开始,直到紧接着clat完成的时间

clat percentiles (usec):
 |  1.00th=[ 4080],  5.00th=[ 5984], 10.00th=[ 7136], 20.00th=[ 8640],
 | 30.00th=[ 9920], 40.00th=[10944], 50.00th=[11840], 60.00th=[12992],
 | 70.00th=[14016], 80.00th=[15296], 90.00th=[17024], 95.00th=[18560],
 | 99.00th=[20864], 99.50th=[21632], 99.90th=[22400], 99.95th=[22912],
 | 99.99th=[23168]
# Completion latency百分数

bw (KB  /s): min=  276, max=  373, per=100.00%, avg=331.54, stdev=18.83
# 带宽(bandwidth)

lat (msec) : 2=0.02%, 4=0.82%, 10=30.32%, 20=66.75%, 50=2.09%
# 30.32%的request延迟在4~10毫秒,66.75%的request延迟在10~20毫秒,以此类推

cpu : usr=0.10%, sys=0.25%, ctx=5057, majf=0, minf=8
# 用户/系统CPU占用率,进程上下文切换(context switch)次数,主要和次要(major and minor)页面错误数量(page faults)。由于测试是配置成使用直接IO,page faults数量应该极少。

IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
# iodepth设置,用来控制同一时刻发送给OS多少个IO。这完全是纯应用层面的行为,和盘的IO queue不是一回事

 submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
 #submit和complete代表同一时间段内fio发送上去和已完成的IO数量

 issued    : total=r=4977/w=0/d=0, short=r=0/w=0/d=0
 # 发送的IO数量

 latency   : target=0, window=0, percentile=100.00%, depth=1
 # Fio可以配置一个延迟目标值,这个值可以调节吞吐量直到达到预设的延迟目标

Run status group 0 (all jobs):

Fio支持把不同的测试聚合

READ: io=19908KB, aggrb=331KB/s, minb=331KB/s, maxb=331KB/s, mint=60013msec, maxt=60013msec
# 汇总输出吞吐量和时间。
# io=表示总共完成的IO数量。在基于时间的测试中这是一个变量,在基于容量的测试中,这个值能匹配size参数。
# aggrb是所有进程/设备的汇总带宽。
# minb/maxb表示测量到的最小/最大带宽。
# mint/maxt表示测试的最短和最长耗时。和io=参数类似,时间值对于基于时间的测试应该能匹配runtime参数,对于基于容量的测试是一个变量。

Disk stats (read/write):
sdd: ios=4969/0, merge=0/0, ticks=59703/0, in_queue=59702, util=99.51%

Python日志监控系统处理日志(pyinotify)

前言

最近项目中遇到一个用于监控日志文件的Python包pyinotify,结合自己的项目经验和网上的一些资料总结一下,总的原理是利用pyinotify模块监控日志文件夹,当日志到来的情况下,触发相应的函数进行处理,处理完毕后删除日志文件的过程,下面就着重介绍下pyinotify

pyinotify

Pyinotify是一个Python模块,用来监测文件系统的变化。 Pyinotify依赖于Linux内核的功能—inotify(内核2.6.13合并)。 inotify的是一个事件驱动的通知器,其通知接口通过三个系统调用从内核空间到用户空间。pyinotify结合这些系统调用,并提供一个顶级的抽象和一个通用的方式来处理这些功能。

  • pyinotify 说百了就是通过 调用系统的inotify来实现通知的
  • inotify 既可以监视文件,也可以监视目录
  • Inotify 使用系统调用而非 SIGIO 来通知文件系统事件。

Inotify 可以监视的文件系统事件包括:

Event NameIs an EventDescription
IN_ACCESSYesfile was accessed.
IN_ATTRIBYesmetadata changed.
IN_CLOSE_NOWRITEYesunwrittable file was closed.
IN_CLOSE_WRITEYeswrittable file was closed.
IN_CREATEYesfile/dir was created in watched directory.
IN_DELETEYesfile/dir was deleted in watched directory.
IN_DELETE_SELFYes自删除,即一个可执行文件在执行时删除自己
IN_DONT_FOLLOWNodon’t follow a symlink (lk 2.6.15).
IN_IGNOREDYesraised on watched item removing. Probably useless for you, prefer instead IN_DELETE*.
IN_ISDIRNoevent occurred against directory. It is always piggybacked to an event. The Event structure automatically provide this information (via .is_dir)
IN_MASK_ADDNoto update a mask without overwriting the previous value (lk 2.6.14). Useful when updating a watch.
IN_MODIFYYesfile was modified.
IN_MOVE_SELFYes自移动,即一个可执行文件在执行时移动自己
IN_MOVED_FROMYesfile/dir in a watched dir was moved from X. Can trace the full move of an item when IN_MOVED_TO is available too, in this case if the moved item is itself watched, its path will be updated (see IN_MOVE_SELF).
IN_MOVED_TOYesfile/dir was moved to Y in a watched dir (see IN_MOVE_FROM).
IN_ONLYDIRNoonly watch the path if it is a directory (lk 2.6.15). Usable when calling .add_watch.
IN_OPENYesfile was opened.
IN_Q_OVERFLOWYesevent queued overflowed. This event doesn’t belongs to any particular watch.
IN_UNMOUNTYes宿主文件系统被 umount
IN_ACCESS,即文件被访问
IN_MODIFY,文件被write
IN_ATTRIB,文件属性被修改,如chmod、chown、touch等
IN_CLOSE_WRITE,可写文件被close
IN_CLOSE_NOWRITE,不可写文件被close
IN_OPEN,文件被open
IN_MOVED_FROM,文件被移走,如mv
IN_MOVED_TO,文件被移来,如mv、cp
IN_CREATE,创建新文件
IN_DELETE,文件被删除,如rm
IN_DELETE_SELF,自删除,即一个可执行文件在执行时删除自己
IN_MOVE_SELF,自移动,即一个可执行文件在执行时移动自己
IN_UNMOUNT,宿主文件系统被umount
IN_CLOSE,文件被关闭,等同于(IN_CLOSE_WRITE | IN_CLOSE_NOWRITE)
IN_MOVE,文件被移动,等同于(IN_MOVED_FROM | IN_MOVED_TO)

pyinotify使用例子

#!/usr/bin/python
# coding:utf-8
 
import os
from pyinotify import WatchManager, Notifier,ProcessEvent,IN_DELETE, IN_CREATE,IN_MODIFY
   
class EventHandler(ProcessEvent):
 """事件处理"""
 def process_IN_CREATE(self, event):
  print  "Create file: %s " %  os.path.join(event.path,event.name)
   
 def process_IN_DELETE(self, event):
  print  "Delete file: %s " %  os.path.join(event.path,event.name)
  
 def process_IN_MODIFY(self, event):
   print  "Modify file: %s " %  os.path.join(event.path,event.name)
   
def FSMonitor(path='.'):
  wm = WatchManager() 
  mask = IN_DELETE | IN_CREATE |IN_MODIFY
  notifier = Notifier(wm, EventHandler())
  wm.add_watch(path, mask,auto_add=True,rec=True)
  print 'now starting monitor %s'%(path)
  while True:
   try:
     notifier.process_events()
     if notifier.check_events():
       notifier.read_events()
   except KeyboardInterrupt:
     notifier.stop()
     break
   
if __name__ == "__main__":
 FSMonitor('/root/softpython/apk_url')

How to Install Windows on RAID 5 with Drives Larger than 2TB

Setting Up the PERC with the DISKPART Tool for Windows Install

This type of RAID array requires a partition using a GUID (Globally Unique Identifier) Partition Table (GPT) to work properly for Windows in UEFI BIOS mode. Windows Setup, however, does not support making a GPT partition in the standard setup dialogue. Diskpart from the Command Prompt must be used to create a GPT partition to allow setup to see the RAID partition and continue setup.


Figure 1

  1. Start the install, and load the PERC H310 drivers via the OS “Load Diver” function. After the driver is loaded the RAID volume will appear in the install to device list (Figure 1).
  2. Press Shift+F10 to bring up a Command Prompt window.
  3. Type DISKPART and press Enter to enter the DISKPART tool. Enter the commands in the following steps as shown in bold and press enter.
  4. DETAIL DISK – This shows a list of volumes seen by the system. Make note of the Volume number for the RAID array.
  5. SELECT DISK=X – X will be the Volume number of the RAID shown in the detail disk report.
  6. CLEAN – Clears the partition information.
  7. CONVERT GPT – Sets the partition to GPT.
  8. EXIT – Exits DISKPART.
  9. Exit the Command Prompt window.
  10. The full RAID volume should show in the device list.
  11. Complete the remainder of the installation process normally. The RAID volume should show as “Windows Boot Manager” in UEFI.

If this was completed correctly, the system should boot normally in UEFI BIOS mode, and allow Windows to install.

Configure IIS to listen on specific IPs

By default IIS will listen for connections on port 80 for any IP bound to the server. This happens even if there are no host headers or bindings set for a specific IP. This can be a problem when trying to run multiple web servers on port 80.

To set IIS to listen on specific IPs follow the instructions below.

Windows Server 2003/IIS 6:

1. This requires the Server 2003 support tools. If this is not already installed it can be downloaded here.

2. Once installed open a command prompt and navigate to the support tools installation folder (default is C:\Program Files\Support Tools).
cd C:\Program Files\Support Tools

3. Stop http.
net stop http /y

4. Use this command to display the current list of IPs:
httpcfg query iplisten

5. By default it will listen on all IPs (0.0.0.0) so we can remove this.
httpcfg delete iplisten -i 0.0.0.0

6. Specify the IP(s) that IIS should listen on. Make sure to update 127.0.0.1 to the desired IP and run the command for each IP IIS should listen on.
httpcfg set iplisten -i 127.0.0.1

7. Start http and test out your sites.
net start http

Windows Server 2008/IIS 7:

1. Open a command prompt and type “netsh”.
netsh

2. Type “http”.
http

3. Enter the following command to display the current list of IPs to listen on. Note if no IPs are displayed like in the below image, IIS will listen on all IPs (default).
show iplisten

4. Use the command below to set IIS to listen on a specific IP. Make sure to replace 127.0.0.1 with the correct IP and run the command again for any additional addresses.
add iplisten ipaddress=127.0.0.1

5. In case you need to delete an IP from this list, use the following command.
delete iplisten ipaddress=127.0.0.1

6. Restart IIS to apply these changes.
iisreset

IIS listening 127.0.0.1 instead 0.0.0.0

I use command:

netsh http show iplisten

and saw 127.0.0.1 in listening list. (But I didn’t add it manually). So I delete it and add 0.0.0.0 instead.

netsh http delete iplisten ipaddress=127.0.0.1

netsh http add iplisten ipaddress=0.0.0.0

Then I restarted iis server.

SSL certificates on Sites with Host Headers

Source:https://blogs.iis.net/thomad/ssl-certificates-on-sites-with-host-headers

Today I got the following question:

“I have two sites (siteV1.mysite.com and sitev2.mysite.com). They listen on the same IP address and port. We generated a certificate for siteV1.mysite.com and SSL is working properly. The problem is that some of our customers use siteV2.mysite.com and they are getting certificate errors. What’s the problem?”

Here is the issue:

There are three pieces of data to uniquely identify an IIS site:

  • The IP address
  • The Port
  • The Host name which HTTP 1.1 clients send as an HTTP request header. 

This IP:Port:Hostname triplet is called a binding. The binding “192.168.1.192:80:myserver” for example represents a site that listens on IP address 192.168.1.192, port 80, host-header myserver

The very first things IIS (HTTP.SYS to be more precise) does when a request comes in is to read the site’s configuration. Connection limits and timeouts are examples of site configuration. The site binding is used to find the right site configuration. The SSL certificate seems to be another great example of site configuration – the SSL certificate is needed to decrypt the encrypted SSL data coming from the client.

And the IIS User Interface certainly makes it appear as if the SSL certificate would be site configuration, too – doesn’t it? In reality however you can’t bind a SSL certificate to a site. The IIS UI is fooling you. But why? 

It’s a chicken and egg problem: The host name is encrypted in the SSL blob that the client sends. Because the host name is part of the binding IIS needs the host name to lookup the right certificate. Without the host name IIS can’t lookup the right site because the binding is incomplete. Without the certificate IIS can’t decrypt the SSL blob that contains the host name. Game over – we are turning in circles. 

What IIS does under the covers is to ignore the host name. IIS binds the certificate to IP:Port and warns you when you try to bind a certificate to the same IP:Port combo with different host names. 

But there is a way if you need two different sites on the same IP:Port. You can accomplish this by getting a certificate that contains both common names, i.e. sitev1.mysite.com and sitev2.mysitem.com. Cert Authorities usually allow more than one so called “common names” in a certificate. By binding the certificate to one of the two sites you won’t not get certificate errors anymore. The client is happy if one of the names in the certificate matches. 

But there is another caveat: you can’t use the IIS7 User Interface to add a host header to an SSL site binding. You have to use command-line tools, do it programmatically or edit applicationhost.config directly. Here is an example and a link how you can it via command-line:

appcmd set site /site.name:”MySite V2″ /+bindings.[protocol=’https’,bindingInformation=’*:443:sitev2.mysite.com’]

And last but not least: with IIS7 you can use the following command to figure out what certificate is bound to a particular IP:Port combination:  
netsh http show sslcert

This command will show the IP:Port binding but also some other SSL settings.

Zabbix通过Smokeping检测网络质量并告警

(一)Smokeping概述
(1) Smokeping是一款用于网络性能监测的开源监控软件,主要用于对IDC的网络状况,网络质量,稳定性等做检测,通过rrdtool制图方式,图形化地展示网络的时延情况,进而能够清楚的判断出网络的即时通信情况。

(2)SmokePing的特点
SmokePing keeps track of your network latency:
Best of breed latency visualisation.(最佳图形展示功能,延时丢包等可以很直观的可视化展现)
Interactive graph explorer.(交互式浏览器图表)
Wide range of latency measurement plugins.(丰富的网络状况测量插件)
Master/Slave System for distributed measurement.(支持主从的分布式部署模式)
Highly configurable alerting system.(自定义报警功能)
Live Latency Charts with the most ‘interesting’ graphs.(漂亮、免费、开源)
Free and OpenSource Software written in Perl written by Tobi Oetiker, the creator of MRTG and RRDtool

(二)技术概述
使用zabbix通过smokeping来检测网络质量zabbix官方论坛有推荐,可以登陆查看下,具体地址如下:https://www.zabbix.com/forum/showthread.php?t=31147
技术关键点有三点:
1,zabbix-trapper:这是一种数据传递方式,不同于zabbix-agent,这种方式定义的item需要使用zabbix-sender来发送数据给zabbix-server

2,zabbix-sender需要的参数:

-z - 指定zabbix server的IP
-p - 指定zabbix server的端口,默认为10051
-s - 指定目标主机,主机名必须是配置中的hostname而不是visible name,切记
-k - 指定key,我们定义的trapper的key,这边便是我们前面定义的trap
-o - 指定要传递的数据

3,使用fping探测各节点丢包率

fping的参数:
-b ping包大小
-c ping的次数   
-p ping间隔,单位ms

(三)具体步骤

(一)在zabbix_server端的配置
1,放开zabbix_server.conf中ExternalScripts的配置并设置为:ExternalScripts=/usr/local/zabbix/externalscripts

[root@localhost externalscripts]# vim /usr/local/zabbix/etc/zabbix_server.conf

### Option: AlertScriptsPath
#       Full path to location of custom alert scripts.
#       Default depends on compilation options.
#
# Mandatory: no
# Default:
# AlertScriptsPath=${datadir}/zabbix/alertscripts
 AlertScriptsPath=/usr/local/zabbix/alertscripts

### Option: ExternalScripts
#       Full path to location of external scripts.
#       Default depends on compilation options.
#
# Mandatory: no
# Default:
# ExternalScripts=${datadir}/zabbix/externalscripts
ExternalScripts=/usr/local/zabbix/externalscripts

2,把zabbix官方推荐的脚本放到ExternalScripts=/usr/local/zabbix/externalscripts/目录下,http://www.mbs-it.pl/inne/zbxsmokeping

[root@localhost externalscripts]# cat zbxsmokeping
#!/bin/bash
# Where is your zabbix server
ZBXSERVER=172.20.66.110
# where is fping tool?
FPING=/usr/sbin/fping
# where is zabbix_sender tool?
ZBXSENDER=/usr/local/zabbix/bin/zabbix_sender
# Where to send ping
IP=$1
# How many ping to send
COUNT=$2
# What interval between ping [ms]
INTERVAL=$3
# How many bytes in one ping
BYTES=$4
# 'Hostname' of the host which will collect data
HOSTNAME=$5

if [ $# -lt 5 ]
 then
     echo
     echo " Not enough parameters"
     echo " Usage: zbxsmokeping <HOST_IP> <NUMBERS_OF_PINGS> <INTERVAL> <BYTES> <TO_WHICH_HOST_SEND_DATA_IN_ZABBIX>"
     echo " Zabbix External Check Item ex.: zbxsmokeping[{HOST.IP},6,1000,68,{HOST.HOST}]"
  exit 2
fi
# debug
# echo $FPING -b $BYTES -c $COUNT -q -p $INTERVAL $IP 2>&amp;1

OUTPUT=`$FPING -b $BYTES -c $COUNT -q -p $INTERVAL $IP 2>&amp;1 | awk '{print $5,$8}' | tr -d "%|," | tr -s " " "/" | awk -F"/" '{print $3,$4,$5,$6}'`
tab=( $OUTPUT )
# debug
#echo $ZBXSENDER -z $ZBXSERVER -p 10051 -s $HOSTNAME -k SmokLoos -o ${tab[0]}
#echo $ZBXSENDER -z $ZBXSERVER -p 10051 -s $HOSTNAME -k SmokLatencyMin -o ${tab[1]}
#echo $ZBXSENDER -z $ZBXSERVER -p 10051 -s $HOSTNAME -k SmokLatencyMax -o ${tab[3]}
#echo $ZBXSENDER -z $ZBXSERVER -p 10051 -s $HOSTNAME -k SmokLatencyAvg -o ${tab[2]}

$ZBXSENDER -z $ZBXSERVER -p 10051 -s $HOSTNAME -k SmokLoos -o ${tab[0]}  -v | grep "Failed 1"
$ZBXSENDER -z $ZBXSERVER -p 10051 -s $HOSTNAME -k SmokLatencyMin -o ${tab[1]} -v | grep "Failed 1"
$ZBXSENDER -z $ZBXSERVER -p 10051 -s $HOSTNAME -k SmokLatencyMax -o ${tab[3]} -v | grep "Failed 1"
$ZBXSENDER -z $ZBXSERVER -p 10051 -s $HOSTNAME -k SmokLatencyAvg -o ${tab[2]} -v | grep "Failed 1"
echo 1

3,给该脚本执行权限,并重启下zabbix_server服务。

[root@localhost externalscripts]# chmod +x zbxsmokeping

[root@localhost externalscripts]

# ll zbxsmokeping -rwxr-xr-x 1 root root 1649 Dec 27 17:38 zbxsmokeping

[root@localhost externalscripts]

# /etc/init.d/zabbix_server restart

至此zabbix_server端配置完成。

Zabbix通过Smokeping检测网络质量并告警
Zabbix通过Smokeping检测网络质量并告警

(二)在浏览器端配置。
1,把官方推荐的模板导入进来,http://www.mbs-it.pl/inne/zbx_export_templates_smokeping.xml

2,把模板链接到需要检测网络质量的站点上

Zabbix通过Smokeping检测网络质量并告警

至此zabbix通过smokeping监控网络质量完成。

用pip命令安装第三方包出现retrying且ssl error问题汇总

今天pip包时一直retrying且报ssl error的错误,我弄了一上午才好,网上有很多解决方案,但是没有pip安装失败的汇总情况,如有同错,请对比以下情况,希望能解决你的问题,也烦请对不足之处指出。

一、

错误:CouldnotfetchURLhttps://pypi.python.org/simple/pytest-xdist/: There wasaproblem confirmingthessl certificate: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocolversion(_ssl.c:590)

原因:python.org已经不支持TLSv1.0和TLSv1.1,需要升级pip,但是pip用不了,所以手动升级

解决方案:

1.mac或者linux操作系统:在终端下执行命令:

curl https://bootstrap.pypa.io/get-pip.py | python3。

2.windows操作系统:从https://bootstrap.pypa.io/get-pip.py下载get-pip.py文件,然后使用python运行这个文件python get-pip.py

二、

错误:Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None))

after connection broken by ‘SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAI

LED] certificate verify failed (_ssl.c:661)’),)’

原因:可能用国外镜像源连接不好

解决方案:

1.mac或者linux操作系统:

修改 ~/.pip/pip.conf (没有就创建一个), 更换 index-url

[global]

index-url = http://pypi.douban.com/simple

2.windows操作系统:

直接在user目录中创建一个pip目录,如:C:\Users\xx\pip,新建文件pip.ini,内容同上

附:镜像源

阿里云 http://mirrors.aliyun.com/pypi/simple/

中国科技大学 https://pypi.mirrors.ustc.edu.cn/simple/

豆瓣 http://pypi.douban.com/simple

Python官方 https://pypi.python.org/simple/

v2ex http://pypi.v2ex.com/simple/

中国科学院 http://pypi.mirrors.opencas.cn/simple/

清华大学 https://pypi.tuna.tsinghua.edu.cn/simple/

三、

错误:Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host=’pypi.org’, port=443): Max retries exceeded with url: /simple/pip/ (Caused by S

SLError(SSLError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)’),)) – skipping

原因:在最新的 pip 版本(>=7)中,使用镜像源时,会提示源地址不受信任或不安全

解决方案1:

pip install –trusted-host pypi.org –trusted-host files.pythonhosted.org 包名

解决方案2:(推荐)

在第二个解决方案中:添加一项配置

[install]
trusted-host=http://pypi.douban.com/simple

Windows Process Activation Service error 5 – Access Denied

解决办法:C:\INETPUB\HISTORY. Under here you will see several folders with a prefix of CFGHISTORY. The folder with the highest revision number will be your latest backup. Copy this file and overwrite the existing file at C:\WINDOWS\SYSTEM32\INETSRV\CONFIG.

I ran into a strange error recently on an Exchange 2013 server. The WWW Publishing Service was stopped. When I tried to start the service it failed on a dependency. A quick check revealed the Windows Process Activation Service (WAS) was stopped. When I tried to start WAS, I received the following error.

Windows could not start the Windows Process Activation Service service on Local Computer Error 13 The data is invalid

Windows could not start the Windows Process Activation Service service on Local Computer. Error 13: The data is invalid.

The Event Viewer was littered with equally cryptic Event IDs, such as WAS 5005 and WAS 5036. 继续阅读

SSL证书格式详解与转换

一般来说,主流的 Web 服务软件,通常都基于 OpenSSL 和 Java 两种基础密码库。

  • Tomcat、Weblogic、JBoss 等 Web 服务软件,一般使用 Java 提供的密码库。通过 Java Development Kit (JDK)工具包中的 Keytool 工具,生成 Java Keystore(JKS)格式的证书文件。
  • Apache、Nginx 等 Web 服务软件,一般使用 OpenSSL 工具提供的密码库,生成 PEM、KEY、CRT 等格式的证书文件。
  • IBM 的 Web 服务产品,如 Websphere、IBM Http Server(IHS)等,一般使用 IBM 产品自带的 iKeyman 工具,生成 KDB 格式的证书文件。
  • 微软 Windows Server 中的 Internet Information Services(IIS)服务,使用 Windows 自带的证书库生成 PFX 格式的证书文件。

继续阅读

puppet-external_ip

External IP fact(s)

Sage Imel sage@sagenite.net

Provides an external_ip4 fact that shows your ipv4 IP address as returned by http://ipv4.icanhazip.com. Provides and an external_ip6 fact that shows your ipv6 IP address as returned by http://ipv6.icanhazip.com.

Useful if you have a host with a dynamic IP address.

Limitations

Ruby doesn’t seem to let you specify which interface Web::HTTP uses, so on a box with multiple interfaces your milage may very.

https://raw.githubusercontent.com/nightfly19/puppet-external_ip/master/lib/facter/external_ip4.rb

require 'net/http'

Facter.add("external_ip4") do
  setcode do
    begin
      target = URI.parse('http://ipv4.icanhazip.com/')
      Net::HTTP.get_response(target.host, target.path).body.chomp
    rescue
      nil
    end
  end
end
require 'net/http'

https://raw.githubusercontent.com/nightfly19/puppet-external_ip/master/lib/facter/external_ip6.rb
Facter.add("external_ip6") do
  setcode do
    begin
      target = URI.parse('http://ipv6.icanhazip.com/')
      Net::HTTP.get_response(target.host, target.path).body.chomp
    rescue
      nil
    end
  end
end