一、Changedetection是什么
ChangeDetection-一个用于监控网页内容变化的开源系统。我们先引用开发者对该系统的介绍
changedetection.io - The best and simplest self-hosted free open source website change detection monitoring and notification service. An alternative to Visualping, Watchtower etc. Designed for simplicity - the main goal is to simply monitor which websites had a text change for free. Free Open source web page change detection
来自Github
翻译成中文:
changedetection.io - 最好和最简单的自托管免费开源网站变更检测监控和通知服务。 Visualping、Watchtower 等的替代方案,为简单而设计 - 主要目标是简单地监控哪些网站更改了文本。
哪些场景会用到呢,官方也给出了示例(括号内为大白话版本):
- 监控产品的价格变化(比如用于淘宝、京东、天猫、亚马逊等电商平台的产品价格监控)
- 政府部门更新(比如监控最新的招投标信息)
- 当您不在他们的邮件列表中时,新软件版本、安全建议(比如跟踪github某项目的最新release版本)
- 有变化的节日(比如监控法定假日?)
- 房源变更(比如某8?)
- 来自政府网站的 COVID 相关新闻(比如本次上海疫情每日的新增病例数)
- 来自他们网站的大学/组织新闻(比如x大学招生动态)
- 检测和监控 JSON API 响应的变化
- JSON API 监控和警报
- 法律和其他文件的变化
- 当文本出现在网站上时通过通知触发 API 调用(比如监控某个一直缺货的商品,在其有库存后发出提醒)
- 使用 JSON 过滤器和 JSON 通知将 API 组合在一起
- 根据 Web 内容的变化创建 RSS 提要
- 监控 HTML 源代码的意外更改,加强您的 PCI 合规性
- 您有一个非常敏感的 URL 列表要监控,并且您不想使用付费的替代方案。 (想白P)
好了言归正传,下面介绍如何在利用宝塔面板搭建这个开源系统。
根据官网的wiki,该系统有多重部署方式,官方推荐的是docker方式,但为了节约vps宝贵的存储资源,我并不想安装在机器上安装docker,所以这里采用另一种方式pip安装。
二、Python环境准备
Changedetection需要python3.7以上版本,所以如果你的python版本低于3.7需要先卸载之前的版本再重新安装新版本。这里需要友情提示一下,宝塔面板其实是自带python2.x的,但记住该版本的python一定不要卸载,否则面板使用有可能出现问题。这里说的卸载低版本是指如果你安装了低版本的python3.x的情况,切记。
(一)、卸载旧版python3.x
例如你之前安装过python3.6,依次执行以下命令进行3.6的卸载
rpm -qa|grep python3|xargs rpm -ev --allmatches --nodeps ##卸载pyhton3
whereis python3 |xargs rm -frv ##删除所有残余文件
卸载完以后我们检查一下系统中现存的python版本
whereis python ##通过此命令查看现有python的位置
如果在结果中只有2.x的话证明旧版的3.6已经卸载成功,接下来安装高版本的python,比如我这里选择安装python3.7.5
(二)、安装python
1、下载及安装
wget https://www.python.org/ftp/python/3.7.5/Python-3.7.5.tgz ##下载安装包
tar -xzvf Python-3.7.5.tgz ##解压
cd Python-3.7.5 ##进入文件夹
./configure -prefix=/usr/local/python3.7.5 ##注意:prefix参数用于指定将Python安装在新目录,防止覆盖宝塔系统默认安装的python2.x
make && make install ##编译
2、 建立软链接(python3的环境变量)
这里建立python3的环境变量,作用就是当我们在终端里输入python3这个命令的时候让系统调用刚刚安装的新版python而不是其他版本,命令如下
rm -rf /usr/bin/python3 ##删除原有链接
ln -s /usr/local/python3.7.5/bin/python3.7 /usr/bin/python3 ##建立新的软连接
3、 建立软链接(pip的环境变量)
这里建立pip3的环境变量,作用就是当我们在终端里输入pip这个命令的时候让系统调用刚刚安装的新版pip工具而不是其他版本,命令如下
rm -rf /usr/bin/pip ##删除原有链接
ln -s /usr/local/python3.7.5/bin/pip3 /usr/bin/pip ##建立新的软连接
执行完之后我们可以用pip -v命令检查下pip版本,如果执行pip命令报错的话可能需要重装一下pip了,命令是
rm -rf /usr/bin/pip ##删除原有链接
python3 get-pip.py --force-reinstall ##强制重装pip
OK,到这里整个环境已经搭建好了,下面进入安装
三、安装ChangeDetection
简直不要太简单,直接在终端里面执行以下命令
pip install changedetection.io
如果你想安装指定的版本可以用下面的格式
pip install changedetection.io==0.39.12 ##安装0.39.12版本
如果想卸载可以使用pip uninstall 这个命令,同样的卸载指定版本的命令是
pip uninstall changedetection.io==0.39.12
安装完以后,程序的源码会出现在python安装目录下,如果不需要改源码的话可以不用可以去找了。
不过想要程序运行还需要指定一个额外的目录,比如我们在/www/wwwroot/下新建一个文件夹ChangeDetection,然后执行以下命令让程序跑起来
changedetection.io -d /www/wwwroot/ChangeDetection -p 5000
这个命令仅仅在终端里运行是不行的,因为关闭终端它就会中断执行。我们需要使用宝塔里的一个插件——Supervisor管理器

直接去插件市场搜索安装,安装好以后打开管理器将这个命令加入进程守护。

程序跑起来之后访问本地5000端口就可以看到UI界面了,不过我们需要的是外部通过域名来访问它。
四、新建网站,域名反代
我们在宝塔里新建一个网站,添加一个用于这套监控系统的域名,如果有强迫症,可以把网站目录设置为刚刚创建的那个文件夹/www/wwwroot/ChangeDetection,这样就不会多出一个以域名为文件名的目录,然后点开网站配置中的反向代理选项卡

发送域名填写$host,目标url填写127.0.0.1:5000后即可使用域名访问到系统的前端页面。

Bingo!部署成功,接下来根据自己需要添加监控网址吧。
额外说明:如果需要为域名添加SSL证书,即用https访问的话,需要在配置反向代理之前开启SSL,如果你先开了反代再去申请ssl证书会报错,不过问题不大,只需要先把反代暂停一下即可。
关于Changedetection的使用方法,会在后面的文章中更新。(Changedetection在检测到网页内容变化以后可以通过多种方式向你发出通知,比如通过邮件提醒、通过Bark向你的iPhone发送提醒等等)
五、搭建Webdriver服务器(2022.8.11更新)
Centos上安装chrome浏览器以及驱动并用selenium调用的话,因为没有界面,changedetection在调用webdriver抓取数据时会各种报错。
最终放弃该方案,转用docker方式拉取changedetection做好的镜像。具体操作如下:
1、 在宝塔的软件商店里搜索并安装最新的”Docker管理器“

2、 在Docker管理器的“镜像管理”选项卡里点击“获取镜像”,“官方库”里输入镜像名称selenium/standalone-chrome-debug:3.141.59,然后点击“获取镜像”

3、 在“容器列表”选项卡里点击“创建容器”,镜像选择刚刚获取到的镜像文件,绑定IP默认0.0.0.0,端口映射4444:4444,目录映射/dev/shm:/dev/shm,勾选”开机自启“后提交即可启动该镜像

这时webdriver已经运营在本地127.0.0.1:4444端口上了,接下来我们去changedetection里配置
六、 配置WEBDRIVER_URL
Changedetection主程序要调用我们刚刚运行的WEBDRIVER必须要让它知道WEBDRIVER的地址,官方wiki里是以docker运行的changedetection,WEBDRIVER_URL是以环境变量的方式进行配置;但我们这里是采用pip方式安装的changedetection,环境变量不好配置,所以选择直接在源码中修改。
找到/usr/local/python3.7.5/lib/python3.7/site-packages/changedetectionio/conten_fetcher.py文件,定位到
self.command_executor = os.getenv("WEBDRIVER_URL", 'http://browser-chrome/wd/hub').strip('"')
直接将WEBDRIVER_URL地址修改为
self.command_executor = os.getenv("WEBDRIVER_URL", 'http://127.0.0.1:4444/wd/hub').strip('"')
修改好以后重启一下Changedetection主程序即可,这时再到网站设置里选择内容抓取方式为WebDriver Chrome/Javascript就不会报错了。
七、 配置Bark推送方式
Changedetection集成了Apprise(项目地址https://github.com/caronc/apprise/)作为通知服务,Apprise从v0.9.9版本起支持Bark推送。
如果使用pip安装Changedetection后发现Apprise版本较低无法支持bark推送,可以用以下命令升级
pip install apprise==0.9.9(Changedetection V0.39.14后的版本无需升级)
升级Apprise后就可以在Changedetection的通知里使用bark://{host}:{port}/{device_key} 格式的通知链接将网站变化内容进行推送了。
文章评论
Hello! I just waanted to ask iif you ver have aany
troule with hackers? My last blg (wordpress) was haced annd I ended uup losing many
months off hard work due to nno dsta backup. Do you have any methods to
prevgent hackers?
Veery quickly thgis wbsite will bee famous aid all blogging annd site-building viewers, due
tto it's pleasant articles
@453 Luo Jia nodded, So you see, with batteries, is there a real way to increase penis size we can go to the field natural ed supplements that work Magnesium Male Enhancement Pills of sophisticated machinery and challenge Toyota, Volkswagen, and even Boeing and Airbus But if we do not have strong enough industrial software support, these can only exist in the imagination ; Summary of Agent Indications and Associated Chemoprevention Trials
Fastidious respond in return oof thijs query with solid argumments
andd explaining all onn thee topc of that.
Good post. I learn somthing neww and cchallenging onn blogs I stumbleupon every day.
It will apways be useful to read content from other wriers and practice something from otther websites.
Can Nature Make Us More Caring
Hey there! Do youu use Twitter? I'd like tto follow you if that would be okay.
I'm undoubtedly enjoying your blog and look forward to neww updates.
I amm sure this article has touched all the internet users, its really really faswtidious piece off writing on building
up neww website.
My coder is trying too conjvince mee too move
too .net froom PHP. I have alwas disliked the idea because of
the expenses. Buut he's tryiong none the less. I've ben ueing Movable-type on several
websutes for aboutt a year andd am anxious aabout swithing too anoyher platform.
I have heard good thinbs about blogengine.net.
Is thee a way I can transfer all mmy wordpress content into it?
Any kind of help would bbe really appreciated!
Helko it's me, I aam also visiting this wweb page on a reguhlar basis, this sitte iss genuinely pleasat annd thhe uers are truly sharing fastidious thoughts.
Heeya i'm for the first time here. I found tnis boarrd and I find It truly useful & iit helped
me out a lot. I hope to give something ack annd help others like you helped me.
With havin so much content and articles ddo you ever run into any issues of plagorism orr copyright infringement?
My website haas a lot off completely unique content I've
either authored myself or outsourced but it
appears a loot off it is popping it up all oover the web without mmy agreement.
Do you know any wahs to help reduce content from bwing
stolen? I'd certainly appreciate it.
I was wondering if youu ever considered changing tthe structure of
yoour site? Its vety well written; I love what youve got to say.
Butt maybe yoou coulod a little more in the wayy of
content so people could connect with it better.
Youve got an awful lot of text for onhly having 1 or two pictures.
Mayybe you could space itt out better?
Informative article, exactly what I was lookiung for.
Thanmk yyou for tthe auspicios writeup.
It iin facht was a amusement account it. Lookk advanced too mor
added agreeable froom you! However, hhow cann wwe communicate?
Excellwnt post. I was cecking continuously thos wewblog aand I'm impressed!
Extremepy useul inhfo specifically the losing phaee :)
I handlke such informjation a lot. I sed tto be looking for this parficular
information ffor a very lengthy time. Tanks andd best of luck.
Sweet blog! I found it while surfing around on Yahoo News.
Do yyou have anny tips onn how to geet listedd in Yahoo News?
I've een trying for a while but I never seem to gett there!
Manyy thanks
Thanks a lot foor sharing tthis with all off us yoou actually underztand what you're talkng about!
Bookmarked. Kindly also discuss wih my site =). We can have
a link alternate contract among us
Today, I went to the beach with my children. I found
a sea shell and ggave itt to my 4 year oldd
daughtedr and said "You can hear the ocean if you put this to your ear." She pllaced thee shell
too heer eaar aand screamed. There was a hermit crwb inhside and iit pinched hher ear.
Shhe never wants tto ggo back! LoL I kbow this is toally
off topic but I haad to tepl someone!
Yes! Finally something abot 73894.
Theese are really great ideas iin oon tthe topiic of blogging.
You haave touxhed somme fatidious points here. Any way keep up
wrinting.
Hi, its fastidsious piece oof writing about merdia print, we all understand
media iis a fantastic sourcee off data.
hello there annd tthank you foor your information – I hae definitely picked up anythig
new rom rignt here. I did however expertyise some technical points using this web site, since I experienced too reload tthe webb site a lot of times previous tto I cohld
get it tto llad correctly. I had ben wondering if
yopur web host is OK? Noot that I amm complaining, but sluggish loadig instances tumes will very
frequently ffect your placement in google aand can damage your high-quality score
iff advertising and marketing witth Adwords. Well I aam addng this RSS to
my mail and ccan look ouut forr much more of your respectivve interesting content.
Make surte you update thjis gain soon.
Nice answer back in return off this difficulty woth solid arguments annd telling all
about that.
Hello there! This postt could nott bbe writtgen any better!
Reading through thi post reninds mme of my old rooom mate!
He alwasys kept taking about this. I will forrward thiss write-up too him.
Pretty sure he will haqve a goood read. Tanks for sharing!
Heyy very inteesting blog!
When I orginally commented I clicked the "Notify me when new comments are added" checkbox and
now each time a cokmment is added I get sevberal e-mails with the same comment.
Is there any way you can remolve me frm that service? Thanks a lot!