Hello,

For this third installement of Proxy grinding to a halt:

Terror, angry users and tired sysadmin join force.... XD.

Seriously. Calling our proxy problem fixed in march (see 'Proxy
grinding to a halt revisited - again' (http://tinyurl.com/69tdmt)) was
apparently premature. Our border manager setup (novell 6.5 sp7 bdm 3.9
sp1 2 cache vol: 15 gig,non nss,purge immidiate, cluster 16k, no long
filename) is still having slow moment it just take a lot more time
getting to them. During these episode one of the cache volume will fill
up and the other will be betewen empty and three quaters full. If we
balance it to our backup it will resume normal operation within 15 to
30 minutes.

The problem seem to be streaming media. If a user start a stream and
leave it open it fills one cache volume and the proxy kills itself
trying to clean it. The problem as been reproduce with a test server.

Now i can't block all streaming site with an acl since some of my users
need to be able to access streaming news sites.

So i tried excluding from cache known streaming extansion like
swf,flv,asf etc... but the proxy keep caching them. So i tried
excluding every thing and still border manager keep caching. Is there
something i'm doing wrong ?


--
Darkinspiration
------------------------------------------------------------------------
Darkinspiration's Profile: http://forums.novell.com/member.php?userid=3181
View this thread: http://forums.novell.com/showthread.php?t=333152

In article <Darkinspiration.3b7gqo@no-mx.forums.novell.com>,
Darkinspiration wrote:
> The problem seem to be streaming media. If a user start a stream and
> leave it open it fills one cache volume and the proxy kills itself
> trying to clean it. The problem as been reproduce with a test server.

That makes sense to me. The proxy is, I think, treating that as a single
cached node, so there is no way for it to split the data to multiple
cached volumes. The question will eventually be how to keep a cache
volume from getting slow when it gets full, I guess.
>
> Now i can't block all streaming site with an acl since some of my users
> need to be able to access streaming news sites.

Understand.
>
> So i tried excluding from cache known streaming extansion like
> swf,flv,asf etc... but the proxy keep caching them. So i tried
> excluding every thing and still border manager keep caching. Is there
> something i'm doing wrong ?

This one sounds like something to kick back to Novell as a defect. Can
you open an incident? It sounds like something needs to fundamentally
change to not cache such a stream (if BMgr can identify it), or to
fundamentally change how full cache volumes are treated. (I could be
wrong - perhaps there is a simple tuning switch to change this. Have you
put in my proxy.cfg file, from tip #63 at the URL below?)


Craig Johnson
Novell Support Connection SysOp
*** For a current patch list, tips, handy files and books on
BorderManager, go to http://www.craigjconsulting.com ***

Yes an incident was open with novell. I have not tried your proxy.cfg on
the test server yet it's on my to do list.

Instead i tried this proxy.cfg given to me by novell support



[MiniWeb Server]
Port-Number=1959
Root-Directory=SYS:\ETC\PROXY\DATA

[MiniWeb Server: Mime Types]
Content-Type: text/html=htm,html
Content-Type: text/plain=txt,text,cla,class
Content-Type: image/gif=gif
Content-Type: image/jpeg=jpg,jpeg,jpe,jfif,pjpeg,pjp
Content-Type: image/tiff=tiff,tif
Content-Type: image/x-xbitmap=xbm
Content-Type: video/x-msvideo=avi
Content-Type: video/quicktime=qt,mov,moov
Content-Type: video/x-mpeg2=mpv2,mp2v
Content-Type: video/mpeg=mpeg,mpg,mpe,mpv,vbs,mpegv
Content-Type: audio/x-pn-realaudio=ra,ram
Content-Type: audio/x-mpeg=mpega,mp2,mpa,abs
Content-Type: audio/x-wav=wav
Content-Type: audio/x-aiff=aif,aiff,aifc
Content-Type: application/x-ns-proxy-autoconfig=pac
Content-Type: text/javascript
Content-Type: application/x-javascript


[Extra Configuration]
AckWithNoDataOnSYN=1
AllowGTCPProxyToUsePort25=1
AllowSecond220Respond=1
CodeRedWorkAround=1
DiscardAcceptRanges=1
DonotCache4ContEncoding=1
DoNotCacheWhenCookieFound=1
DoNotCreateFullyQualifiedHostNames=0
DoNotResolveNamesBeforeGoingThruHierarchy=1
DoNotSaveMemoryCacheDuringUnload=1
DoNotSendExtraCRLF=1
DonotSendIPToACL=1
EnableActiveFTP=1
EnableICSPassThruFix=1
EnableIncomplete302ResponseFix=1
EnableNoCachePassThru=1
HTTPSAuthenticationSwitch=0
IcpBypassOrigin=1
IgnoreContentLengthCheck=1
IgnoreDuplicateChill=1
Line_Terminator=CR
new302Redirect=1
NoDummySlashUpstream=1
noRetryIfPragmaNoCacheHeaderPresent=1
OC_IgnoreContentLengthFlag=1
#PassContentLength=0
ResBadAddressLoopBreak=1
#ResolveProxyIPAddress=1
RestartTimeoutAfterEverySend=1
SCacheDestroyYieldInterval=200
ScanVirusPatterns=1
SendHTTP11Request=1
SkipHttpReplyHeaderCaseChange=1
TransparentProxySupportsVirtualServers=1
TreatLeftArrowAsHeaderBodySeparator=0
TurnOffPersistantPassThru=1
UseSimplifiedErrorPage=1


[TransparentHTTPS]
HTTPSPort1=443

[BM Cookie]
BM_Forward_Cookie=0

[Object Cache]
cut thru no CLH length=0
disk management factor=4000

[Log Format]
Delimiter-Character=space

[HTTP Streaming]
ResetOriginServerConnAfterClientReset=1

[Virus Pattern Configuration]
MaxNoOfVirusPatterns=28
NoOfVirusPatterns=28
PatternSize=16
EnableAutoPatternUpdate=1
PatternStartOffset=1
VirusPattern0=scripts/..%252f.
VirusPatternoffset10=0
VirusPatternvalue10=0
VirusPatternoffset20=0
VirusPatternvalue20=0
VirusPatternorigLength0=57
VirusPattern1=scripts/..%c1%1c
VirusPatternoffset11=0
VirusPatternvalue11=0
VirusPatternoffset21=0
VirusPatternvalue21=0
VirusPatternorigLength1=58
VirusPattern2=scripts/..%c0%2f
VirusPatternoffset12=0
VirusPatternvalue12=0
VirusPatternoffset22=0
VirusPatternvalue22=0
VirusPatternorigLength2=58
VirusPattern3=scripts/..%c0%af
VirusPatternoffset13=0
VirusPatternvalue13=0
VirusPatternoffset23=0
VirusPatternvalue23=0
VirusPatternorigLength3=58
VirusPattern4=scripts/..%%35c.
VirusPatternoffset14=0
VirusPatternvalue14=0
VirusPatternoffset24=0
VirusPatternvalue24=0
VirusPatternorigLength4=57
VirusPattern5=scripts/root.exe
VirusPatternoffset15=0
VirusPatternvalue15=0
VirusPatternoffset25=0
VirusPatternvalue25=0
VirusPatternorigLength5=33
VirusPattern6=MSADC/root.exe?/
VirusPatternoffset16=0
VirusPatternvalue16=0
VirusPatternoffset26=0
VirusPatternvalue26=0
VirusPatternorigLength6=31
VirusPattern7=d/winnt/system32
VirusPatternoffset17=0
VirusPatternvalue17=0
VirusPatternoffset27=0
VirusPatternvalue27=0
VirusPatternorigLength7=41
VirusPattern8=c/winnt/system32
VirusPatternoffset18=0
VirusPatternvalue18=0
VirusPatternoffset28=0
VirusPatternvalue28=0
VirusPatternorigLength8=41
VirusPattern9=_mem_bin/..%255c
VirusPatternoffset19=0
VirusPatternvalue19=0
VirusPatternoffset29=0
VirusPatternvalue29=0
VirusPatternorigLength9=78
VirusPattern10=_vti_bin/..%255c
VirusPatternoffset110=0
VirusPatternvalue110=0
VirusPatternoffset210=0
VirusPatternvalue210=0
VirusPatternorigLength10=78
VirusPattern11=msadc/..%255c../
VirusPatternoffset111=0
VirusPatternvalue111=0
VirusPatternoffset211=0
VirusPatternvalue211=0
VirusPatternorigLength11=106
VirusPattern12=scripts/..%%35%6
VirusPatternoffset112=0
VirusPatternvalue112=0
VirusPatternoffset212=0
VirusPatternvalue212=0
VirusPatternorigLength12=59
VirusPattern13=scripts/..%25%35%
VirusPatternoffset113=0
VirusPatternvalue113=0
VirusPatternoffset213=0
VirusPatternvalue213=0
VirusPatternorigLength13=61
VirusPattern14=scripts/..%255c..
VirusPatternoffset114=0
VirusPatternvalue114=0
VirusPatternoffset214=0
VirusPatternvalue214=0
VirusPatternorigLength14=57
VirusPattern15=scripts/..%c1%9c.
VirusPatternoffset115=0
VirusPatternvalue115=0
VirusPatternoffset215=0
VirusPatternvalue215=0
VirusPatternorigLength15=58
VirusPattern16=scripts/root.exe
VirusPatternoffset116=0
VirusPatternvalue116=0
VirusPatternoffset216=0
VirusPatternvalue216=0
VirusPatternorigLength16=81
VirusPattern17=scripts/httpodbc
VirusPatternoffset117=0
VirusPatternvalue117=0
VirusPatternoffset217=0
VirusPatternvalue217=0
VirusPatternorigLength17=30
VirusPattern18=MSADC/root.exe?/
VirusPatternoffset118=0
VirusPatternvalue118=0
VirusPatternoffset218=0
VirusPatternvalue218=0
VirusPatternorigLength18=79
VirusPattern19=MSADC/httpodbc.d
VirusPatternoffset119=0
VirusPatternvalue119=0
VirusPatternoffset219=0
VirusPatternvalue219=0
VirusPatternorigLength19=28
VirusPattern20="c/httpodbc.dll H"
VirusPatternoffset120=0
VirusPatternvalue120=0
VirusPatternoffset220=0
VirusPatternvalue220=0
VirusPatternorigLength20=24
VirusPattern21=d/winnt/system32
VirusPatternoffset121=0
VirusPatternvalue121=0
VirusPatternoffset221=0
VirusPatternvalue221=0
VirusPatternorigLength21=92
VirusPattern22="d/httpodbc.dll H"
VirusPatternoffset122=0
VirusPatternvalue122=0
VirusPatternoffset222=0
VirusPatternvalue222=0
VirusPatternorigLength22=24
VirusPattern23=scripts/..%255c.
VirusPatternoffset123=0
VirusPatternvalue123=0
VirusPatternoffset223=0
VirusPatternvalue223=0
VirusPatternorigLength23=108
VirusPattern24=scripts/.%255c..
VirusPatternoffset124=0
VirusPatternvalue124=0
VirusPatternoffset224=0
VirusPatternvalue224=0
VirusPatternorigLength24=39
VirusPattern25=scripts/..%252f.
VirusPatternoffset125=0
VirusPatternvalue125=0
VirusPatternoffset225=0
VirusPatternvalue225=0
VirusPatternorigLength25=116
VirusPattern26=scripts/..%252f.
VirusPatternoffset126=0
VirusPatternvalue126=0
VirusPatternoffset226=0
VirusPatternvalue226=0
VirusPatternorigLength26=39
VirusPattern27=default.ida?XXXX
VirusPatternoffset127=0
VirusPatternvalue127=0
VirusPatternoffset227=0
VirusPatternvalue227=0
VirusPatternorigLength27=38

It does not seem to resolve the issue. I was trying to follow up one
another of theyre suggestion: to disable caching witch i tried to do.
However bm39 sp1 does not seem to like my exclusions. And so i post
here to rant.


--
Darkinspiration
------------------------------------------------------------------------
Darkinspiration's Profile: http://forums.novell.com/member.php?userid=3181
View this thread: http://forums.novell.com/showthread.php?t=333152

I suppose you could try manually purging the cache volumes each
night... (Using a cron job to unload proxy, load proxy -cc).

Craig Johnson
Novell Support Connection SysOp
*** For a current patch list, tips, handy files and books on
BorderManager, go to http://www.craigjconsulting.com ***

The main problem with this is that i have people using this resource
around the clock. Since the main corporative app is web based and
stored on distant web server i can't just shutdown the proxy for no
reason it breaks sessions. To solve this problem we installed border on
two server using cluster service to share ip address it dosent solve the
session breaking but downtime is minimal. BUT if i do a proxy -cc it
will rebind the cluster address to itself. If i do it on the wrong
server everything goes down. I don't really like this idea. However i
can't see any other workaround.

I'm putting your config file to the test to see if i get a better
result.


--
Darkinspiration
------------------------------------------------------------------------
Darkinspiration's Profile: http://forums.novell.com/member.php?userid=3181
View this thread: http://forums.novell.com/showthread.php?t=333152

In article <Darkinspiration.3bh3yn@no-mx.forums.novell.com>,
Darkinspiration wrote:
> I don't really like this idea. However i
> can't see any other workaround.
>
You could, I suppose, use a cluster script that loads proxy -cc...

I normally, for several reasons, rename proxy.nlm to proxy1.nlm on a
Bmgr cluster server. I don't want proxy to autoload. I then startbrd
and load aclcheck in autoexec.ncf. I use unload proxy1 and load proxy1
as the BMgr startup script, so that only proxy is stopped and started.
You could use this method and load proxy1 -cc to make this a little
easier.

Craig Johnson
Novell Support Connection SysOp
*** For a current patch list, tips, handy files and books on
BorderManager, go to http://www.craigjconsulting.com ***

well our cluster is a little different. The ressources are ip address.
We load the secondary address no arp on server boot then we load border
manager and we delete the secondary ipaddress just before loading the
cluster service. The address are added or deleted depending on where
the resource is. It's faster then reloading proxy.nlm. That means
however that when proxy.nlm is loaded or reloaded it rebind it's
configured ipaddress. If the ressource is not on the same server as the
loaded proxy i have a duplicate ip situation.

I could modify the setup so that every time the resources is migrated
it unload proxy.nlm and delete the address and reload proxy.nlm -cc. It
does add to the migration time however.

I would prefer another solution either caching exclusion or proxy.cfg
fix.

By the way your proxy.cfg seem to fix the bug somehow. I need to test
it more but i let a stream in my proxy server for three days and it
didn't slow down.


--
Darkinspiration
------------------------------------------------------------------------
Darkinspiration's Profile: http://forums.novell.com/member.php?userid=3181
View this thread: http://forums.novell.com/showthread.php?t=333152

Why not give my proxy1.nlm idea a test? I've had great success doing
it that way for a couple of years now. Just have to remember to rename
proxy.nlm after doing a BMgr patch update.

If the cache volumes are empty when proxy1 needs to load, it should
load in less than 1 second.

Craig Johnson
Novell Support Connection SysOp
*** For a current patch list, tips, handy files and books on
BorderManager, go to http://www.craigjconsulting.com ***

Did you have any luck? I believe this is the source of my problems as
well.


--
If something I said has two meanings, and one of those pisses you off, I
meant the other one...
------------------------------------------------------------------------
tmcnicho's Profile: http://forums.novell.com/member.php?userid=2438
View this thread: http://forums.novell.com/showthread.php?t=333152