Unable to sync?

Sync Blizzards' Armory data with WoWRoster (addon depreciated no longer works see ApiSync)

Moderators: Ulminia, poetter

Unable to sync?

Postby Samadhi » Sat Feb 23, 2008 4:17 am

Until recently everything was working well, but suddenly I can't get anybody to sync. I'm not sure if it's the code, my configuration, or just the Armory being its usual flaky self. I just did a fresh install of the [372], but things are still funky. :(

Is anybody else having trouble?

Roster Warning line 1026: UrlGrabber Error [CURL]: connect() timed out!
lib/functions.lib.php Roster Warning line 1026: UrlGrabber Error [CURL]: couldn't connect to host
lib/functions.lib.php Roster Warning line 1026: UrlGrabber Error [CURL]: couldn't connect to host
Image
User avatar
Samadhi
Roster AddOn Dev
Roster AddOn Dev
 
Posts: 114
Joined: Mon Jul 17, 2006 10:38 pm
Location: NJ, USA

Unable to sync?

Postby moofa » Sat Feb 23, 2008 5:51 am

Having the same issues from multiple locations, 155mbit server's located in multiple places each server is using different software however they are all GNU products,

Apache 1.3
Litehttpd

and mainly Apache 2.2.8

All cURL and Curl Wrappers are installed on these boxes' however the sync fails.

I'm pretty sure AT&T's new bandwidth uplinks may have something to do with this, such as packet filtering massive connections from the same IP/14 block to prevent potential ddos attacks.

-2cents
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Unable to sync?

Postby DrSuSE » Tue Feb 26, 2008 12:54 pm

I don't know what the issue is, or what information to report to help, but my sync'ing starts out fine... gets done 10-15 or so characters, then all failures. During the failures I can see access the WoW Armory just fine.
DrSuSE
WR.net Apprentice
WR.net Apprentice
 
Posts: 16
Joined: Mon Nov 13, 2006 11:23 pm

Unable to sync?

Postby poetter » Tue Feb 26, 2008 4:06 pm

Well, this seems to be new ddos filter at blizzard. No idea yet how to handle this.
Image
User avatar
poetter
Roster AddOn Dev
Roster AddOn Dev
 
Posts: 462
Joined: Sat Jun 30, 2007 9:41 pm
Location: Germany/Hamburg

Re: Unable to sync?

Postby DrSuSE » Tue Feb 26, 2008 7:33 pm

poetter wrote:Well, this seems to be new ddos filter at blizzard. No idea yet how to handle this.


Ahh, that makes sense.
DrSuSE
WR.net Apprentice
WR.net Apprentice
 
Posts: 16
Joined: Mon Nov 13, 2006 11:23 pm

Unable to sync?

Postby Kruskk » Fri Feb 29, 2008 2:21 am

well this answers my question >_<
Image
User avatar
Kruskk
WR.net Apprentice
WR.net Apprentice
 
Posts: 8
Joined: Fri Jul 07, 2006 10:45 pm
Location: Lexington, KY

Unable to sync?

Postby Samadhi » Fri Feb 29, 2008 2:32 am

I've noticed a couple posts on the official forums from other people asking about this, so it's not just us who are having the problem.

Here's hoping they work something out. I'd hate to have to go back to pestering my guildies to update their profiles again. ;)
Image
User avatar
Samadhi
Roster AddOn Dev
Roster AddOn Dev
 
Posts: 114
Joined: Mon Jul 17, 2006 10:38 pm
Location: NJ, USA

Re: Unable to sync?

Postby Kruskk » Fri Feb 29, 2008 4:20 pm

i hope not but we know blizzard. they would rather the players be using their stuff than some 3rd party work. >_<
Image
User avatar
Kruskk
WR.net Apprentice
WR.net Apprentice
 
Posts: 8
Joined: Fri Jul 07, 2006 10:45 pm
Location: Lexington, KY

Unable to sync?

Postby Keviel » Fri Feb 29, 2008 5:17 pm

Very true unfortunately, despite the fact that their stuff is slow, impossible to depend on, lacks 90% of the features and things the community likes to use. They can't even keep up their own fora on a stabile basis.

Oh well, here's to hoping Vivendi some day gets crushed and replaced by actual Blizzard people.
PHP: 5.2.1 / MySQL: 5.0.27 / Apache: 2.0.52
Keviel
WR.net Apprentice
WR.net Apprentice
 
Posts: 60
Joined: Fri Oct 19, 2007 7:53 pm
Location: Copenhagen, Denmark

Re: Unable to sync?

Postby curtana » Mon Mar 03, 2008 3:57 am

Can I suggest that Sync process has a configurable timer between downloads?

This means that perhaps we can set a character retrieval delay to fox the DDoS security.

Also, perhaps setiings like a "linear' or "round robin" on grabbing the infor could be good. Linear would be from the top down, Round Robin would be a progressive download feature.

Just a couple of thoughts.
curtana
WR.net Apprentice
WR.net Apprentice
 
Posts: 1
Joined: Fri Feb 29, 2008 10:17 am

Re: Unable to sync?

Postby moofa » Mon Mar 03, 2008 1:11 pm

poetter wrote:Well, this seems to be new ddos filter at blizzard. No idea yet how to handle this.


Just add a timer, very simple with php to allow users to set the time per character and reduce your ddos like flood of connections to blizzard.

My personal fix was to deploy multiple ips at once but not all users own multiple C class's.
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Re: Unable to sync?

Postby moofa » Mon Mar 03, 2008 1:31 pm

moofa wrote:
poetter wrote:Well, this seems to be new ddos filter at blizzard. No idea yet how to handle this.


Just add a timer, very simple with php to allow users to set the time per character and reduce your ddos like flood of connections to blizzard.

My personal fix was to deploy multiple ips at once but not all users own multiple C class's.



Just to add aswell, you should make it send a user agent like this, allowing it to pass through the filter quicker.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20080214 Firefox/3.0.0

That will solve more problems then you would think...
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Re: Unable to sync?

Postby Phluxed » Tue Mar 04, 2008 4:02 am

moofa wrote:
moofa wrote:
poetter wrote:Well, this seems to be new ddos filter at blizzard. No idea yet how to handle this.


Just add a timer, very simple with php to allow users to set the time per character and reduce your ddos like flood of connections to blizzard.

My personal fix was to deploy multiple ips at once but not all users own multiple C class's.



Just to add aswell, you should make it send a user agent like this, allowing it to pass through the filter quicker.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20080214 Firefox/3.0.0

That will solve more problems then you would think...


Very very good suggestion.

Looking forward to the next release!
Phluxed
WR.net Apprentice
WR.net Apprentice
 
Posts: 12
Joined: Tue Feb 26, 2008 4:13 am

Re: Unable to sync?

Postby moofa » Wed Mar 05, 2008 9:23 am

moofa wrote:
moofa wrote:
poetter wrote:Well, this seems to be new ddos filter at blizzard. No idea yet how to handle this.


Just add a timer, very simple with php to allow users to set the time per character and reduce your ddos like flood of connections to blizzard.

My personal fix was to deploy multiple ips at once but not all users own multiple C class's.



Just to add aswell, you should make it send a user agent like this, allowing it to pass through the filter quicker.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.7) Gecko/20080214 Firefox/3.0.0

That will solve more problems then you would think...



Code: Select all
    function _setUserAgent( $armory ) {
      global $roster;

      $userAgent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; ';
      switch ( $roster->config['locale'] ) {
         case 'enUS': $userAgent .= 'en-US';
            break;

         case 'deDE': $userAgent .= 'de-DE';
            break;

         case 'frFR': $userAgent .= 'fr-FR';
            break;

         case 'esES': $userAgent .= 'es-ES';
            break;
      }

      $userAgent .= '; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1';
      $armory->setUserAgent($userAgent);

   }


The ddos firewall is reading the user agent right here.

I have circumvented this problem using 12 ips to go at once and using $rand() to choose other agents per request(ip)

Simple fix, plus who use's firefox 2.0.0.1 That browser has XSS expliots, and blizzards firewall does indeed filter it after 4 connections.
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Unable to sync?

Postby moofa » Wed Mar 05, 2008 9:42 am

Just to add, the user agent alone does not fix this on a single ip server.

However a cache can be created by syncing over a long period of time.

Interval has to be very high, 4-8 minutes per 1 or your getting locked out while socketing to the server, Ironically if you lynx wowarmory.com with the same locked out IP address you will connect with no issues.

The problem lays in the method currently being used to grab information, Async- is like a SYN flood to the server then ATT&T's firewall kicks in and ploop you get filtered data, and if you debug the data, the information is actually there however, you get malformed reply headers to c**k block the armory call's.

Blizzard is cheap and out bound bandwidth from ATTENS is expensive ( very ) We use it :)

@poetter if you need any help or a clean workspace just PM me.
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Next

Return to ArmorySync - Depreciated

Who is online

Users browsing this forum: No registered users and 1 guest

cron