Content Filtering Database Management Functions at WEM
SFMDB/SFMDB-INC files
from the MCRDBS on a time-configurable scheduleConfiguring Test URL for Blacklisting
Flow diagram of Content Filtering and Blacklisting
Flow Explanation
NCMEC & iwffiles.WEM
directly fetches URL information from NCMEC and
Iwf bin
files from MCRDBS. MCRDBS polls the iwf server to fetch the iwf
bin files.Directory Structure and File Format for CF and BL under the WEM Server Directory
File Formats
Sfmdb flash/cf/cfdatabases/sfmdb
sfmdb<timestamp>_<vendorno>_<majorno>_<minorno>.pub
sfmdb20130805151040_1_023_1697.pub
Sfmincdb flash/cf/cfdatabases/sfmincdb
sfmincdb<timestamp>_<vendorno>_<majorno>_<minorno>.pub
sfmincdb20130805151040_1_023_1697.pub
Optcmdb flash/cf/cfdatabases/optcmdb/
optcmdb<timestamp>_<vendorno>_<majorno>_<minorno>.bin
optcmdb20130805151040_1_023_1697.bin
Optcmincdb flash/cf/cfdatabases/optcmincdb
optcmincdb<timestamp>_<vendorno>_<majorno>_<minorno>.bin
optcmincdb20130805151040_1_023_1697.bin
Ncmec flash/cf/blacklistdb/optcmbldb/ncmec
ncmec<timestamp>_<uniqueno>.bin
ncmec20130806035404_43.bin
Iwf: flash/cf/blacklistdb/optcmbldb/iwf Format: ncmec<timestampepoch>_<uniqueno>.bin Example: iwf1374696564_83.bin
Optcmbldb /flash/cf/blacklistdb/optcmbldb
optcmbldb<timestamp>_<uniqueno>.bin
optcmbldb20130731234542_60.bin
Wblistdb flash/cf/cfdatabases/wblistdb
Sfmdb-latest: ( This
folder is used for internal purpose during conversion of starent
specific format to binary file) flash/cf/cfdatabases/ sfmdb-latest/
Sfmincdb-haspath:
( This folder is used for internal purpose during conversion of
starent specific format to binary file) flash/cf/cfdatabases/ sfmdb-latest/
Statistics for Content Filtering and Blacklisting
<wem installation
dir>/server/flash/cf/statistics
cf.cfg file located at <wem
installation>/<server>/etc/cf.cfg. This
time interval is configured in seconds. After you configure the
cfstat time interval, statistics is generated every cfstat interval
for Content Filtering. For example, if you configure CFStatsInterval = 300,
then WEM collects the following statistics every 300 seconds. The
default value, min value, and maximum value for the cfstats is as
follows: Content Filtering Statistics
Blacklisting Statistics
Configuring WBLIST by Operator
[root@pnqaextappsucs210-05 tools]# ./wblist
-help
Usage :
-f <path> : path
is wblist source file path
-o <output path>
: WBLIST will be created at <path>/wblist.pub
: [default : current path]
-v <version> :
WBLIST version number : [default 1]
-h : Help--------------------------------------------------------
[root@pnqaextappsucs210-05 tools]#Example:[root@pnqaextappsucs210-05
tools]#
[root@pnqaextappsucs210-05
tools]# ./wblist -f wblist.txt -o /export/home/SQA/ems/server/flash/cf/cfdatabases/wblistdb/ -v 2
Setting WBLIST file
path : /export/home/SQA/ems/server/flash/cf/cfdatabases/wblistdb//wblist.pub
--------------------------------------------------------
Source File: wblist.txtWBLIST
file path:
/export/home/SQA/ems/server/flash/cf/cfdatabases/wblistdb//wblist.pub--------------------------------------------------------
[root@pnqaextappsucs210-05
tools]#
This wblist.pub automatically
gets merged with the SFMDB if created at
<EMS Server>/ server/flash/cf/cfdatabases/wblistdb/
Category List
| Category | Description |
|---|---|
|
ABOR
|
Abortion
|
|
ADULT
|
Adult Related Material
|
|
ADVERT
|
Advertising site
|
|
ANON
|
Anonymizer
|
|
ART
|
Art, Museums
|
|
AUTO
|
Automotive site
|
|
BACKUP
|
Online Backup and file
storage
|
|
BLACK
|
Inappropriate Content
|
|
BLOG
|
Blogging
|
|
BUSI
|
Business
|
|
CAR
|
Career, Job Search
|
|
CDN
|
Content Delivery Network
|
|
CHAT
|
Chatting site
|
|
CMC
|
Virtual Community
|
|
CRIME
|
Criminal Skills
|
|
CULT
|
Cult
|
|
DRUG
|
Drug
|
|
DYNAM
|
Dynamic site, Wireless
|
|
EDU
|
Educational site
|
|
ENERGY
|
Energy
|
|
ENT
|
Entertainment, Music
|
|
FIN
|
Finance
|
|
FORUM
|
Forum and Messageboard
|
|
GAMB
|
Gambling
|
|
GAME
|
Gaming
|
|
GLAM
|
Glamour
|
|
GOVERN
|
Government site
|
|
HACK
|
Hacking
|
|
HATE
|
Hate Site, Hate Speech
|
|
HEALTH
|
Health site
|
|
HOBBY
|
Hobby
|
|
HOSTS
|
Hosting site
|
|
KIDS
|
Kids
|
|
LEGAL
|
Legal, Law site
|
|
LIFES
|
Lifestyle
|
|
MAIL
|
Webmail
|
|
MIL
|
Military
|
|
NEWS
|
News site
|
|
OCCULT
|
Occult
|
|
PEER
|
File Sharing
|
|
PERS
|
Personals and Dating
|
|
PHOTO
|
Photography
|
|
PLAG
|
Plagiarism
|
|
POLTIC
|
Politics
|
|
PORN
|
Pornography, Nudism,
Naturism
|
|
PORTAL
|
Portal site
|
|
PROXY
|
Proxy, Test
|
|
REF
|
Reference site
|
|
REL
|
Religion
|
|
SCI
|
Science
|
|
SEARCH
|
Search, Web Search
|
|
SHOP
|
Online Shopping
|
|
SPORT
|
Sport
|
|
STREAM
|
Streaming Media
|
|
SUIC
|
Suicide
|
|
SXED
|
Sexual Education
|
|
TECH
|
Technology and Telecommunication
|
|
TRAV
|
Travel
|
|
VIOL
|
Violence
|
|
VOIP
|
Internet Telephony
|
|
WEAP
|
Weapons
|
|
WHITE
|
Clean Content
|
|
UNKNOW
|
Unknown URLs which are
not present in the optimized content category database
|
Glossary Terms