Wallet sync is very slow

Wellenreiter

@wrapper said in Wallet sync is very slow:

Just reading your post again, to see if I could help further.

The action of you anti virus seems weird, why is writing to a log file being picked up as a virus? , what is the name of the virus checker? Can you try turning it off?

I’d be very interested in the comparison time to synchronize other coins e.g. Litecoin or Bitcoin Blockchain.

I think the virus check simply checks the log file on every write, even without flagging it as spam. This causes higher CPU load and also may slow down the Sync in return.

@looki :
Can you configure your virus check to check executables and libraries only? That way it would not scan/tail the wallet log file

wrapper

@Wellenreiter also noticed you have no incoming connections : which could be a firewall or UPNP off problem.

looki

Hi guys, thank you for your replies.
I am using the latest version 0.6.9.1, x64 windows edition.
Yep I used the shrinkdebuglog option for not overloading the system too much.
As wellenreiter wrote, the anti-virus recognize massive writing data to disk regardless of network use, which looks suspicious thats why it’s use high cpu.
I excluded the feather coin wallet from the antivirus daemon. Executing feathercoin wallet doesn’t executes the anti-virus on-access daemon.
Wrapper: I checked with some different wallets with huge block chain, the average syncing process took about hour and half. Firewall is not blocking the feather coin wallet + Upnp option is enabled.

The source code of other mature walltes are open and published at github. maybe it would be useful to compare to find bottlenecks, and fix them.

thanks

wrapper

@looki said in Wallet sync is very slow:

The source code of other mature walltes are open and published at github. maybe it would be useful to compare to find bottlenecks, and fix them.

Feathercoin is based on the same code as Litecoin and Bitcoin.

looki

@wrapper said in Wallet sync is very slow:

@looki said in Wallet sync is very slow:

The source code of other mature walltes are open and published at github. maybe it would be useful to compare to find bottlenecks, and fix them.

Feathercoin is based on the same code as Litecoin and Bitcoin.

Yep, thats why it’s suspicious that basically same piece of software act different, despite number of nodes( for instance i fully synced vertCoin wallet while connected to 2 nodes only in about an hour)

looki

In addition, there is a weird issue, everytime I close the wallet a save dialog is prompted asking to backup the wallet. Is it a configuration option? Or something messed up with my wallet?
thanks

wrapper

The backup option was something put in by Lizhi, were removing that for future versions, it is there to remind people to back-up if they spent coins. I just use the other shut down.

Our preferred option is to work on moving to 0.11.x versions, which cure some of the sync issues. Certainly, the fact that syncing seems slow for some people, has ment we are looking at testing a couple of ideas (of what might be slowing you down.) That would be for a 0.9.6.2 intermediate release.

Can you check the getpeerinfo again? Just to see if you now have some in connections?

wrapper

Can you try this fix (potential)?

feathercoin-qt -maxorphantx=2000

I think I’ve found the (main) cause of the issue slow sync from scratch and why it only effects some people, it effects other coins occasionally as well.

The max orphan count in memory is set too low, 100, it has been increased to 750.

It can be reset once a full sync has completed.

There are a few longer orphan chains to pass on a full Feathercoin long sync, most time they will go through as the real block is in memory, however if it is not the 750 is cleared and orphans reloaded.

a feature -maxorphan=<n> was introduced in this commit

https://github.com/bitcoin/bitcoin/issues/4353

That appears to be -maxorphantx=<n>

main.cpp

http://forum.feathercoin.com/topic/8918/dev-documenting-feathercoin-specific-software-settings-part-1/12

    // DoS prevention: do not allow mapOrphanTransactions to grow unbounded
 -  unsigned int nEvicted = LimitOrphanTxSize(MAX_ORPHAN_TRANSACTIONS);
 + unsigned int nMaxOrphanTx = (unsigned int)std::max((int64_t)0, GetArg("-maxorphantx", DEFAULT_MAX_ORPHAN_TRANSACTIONS));
  + unsigned int nEvicted = LimitOrphanTxSize(nMaxOrphanTx);

Max orphan transactions updated (from LTC)

AcidD

@wrapper said in Wallet sync is very slow:

Can you try this fix (potential)?
feathercoin-qt -maxorphantx=2000

Ohhh interesting. Trying this now.

Good find :slight_smile:

wrapper

I haven’t tested it from scratch, but does’n’t seem to be stalling. Down from 1 year 1 week to 50 weeks (30mins).

Seen a couple of long orphan chains go past, none greater than 750 tho’

AcidD

@wrapper

I’m at 2 years and 1 week. still checking and trying a few things

wrapper

Can you try feathercoin-qt -maxorphanblocks=4000 ?

This is the one we want.

Wellenreiter

@looki said in Wallet sync is very slow:

In addition, there is a weird issue, everytime I close the wallet a save dialog is prompted asking to backup the wallet. Is it a configuration option? Or something messed up with my wallet?
thanks

You get that message box, if you close the wallet using the ‘file->quit’ menu entry.

If you close the wallet by clicking the ‘close window’ symbol - mostly an ‘x’ in the very upper right corner- you can leave without saving the wallet.

looki

@wrapper said in Wallet sync is very slow:

Can you try feathercoin-qt -maxorphanblocks=2000 ?

This is the one we want.

Hi wrapper,
Still, can’t see any difference. The weirdest thing is at the moment of starting the wallet, the sync process goes smooth for some seconds(about 30) and then it stalls without any progress.
Typically the “Network Traffic” tab shows progress until about 1-2MB.

![Network Traffic Tab Screenshot] (http://imghost.io/image/HivjX)

wrapper

From our experiments overnight, adding -maxorphanblocks=4000 stopped it stalling at > 750 orphans chains (i.e. need a reset). It is running slower than we think it should, so I’m continuing to investigate.

I’ve been looking at the debug log with /tails -f debug.log , which shows whats being written, in a linux terminal. Nothing shows particularly so far.

taurus

@Wellenreiter said in Wallet sync is very slow:

The initial synchronization may take a day, which is a one time event.

I’m having a similar issue. Been running synch for two days (!) now and still got less than 50% (around block 830,000). I’m running v0.9.6.1 on macMini over fast internet connection. Debug window shows 8 outgoing connections. Synch often gets stuck and needs to be restarted, like every 30 min to a couple of hours. Very frustrating experience. :(

Any suggestions?

Wellenreiter

if you can see a lot of ‘orphan block xxx’ where xxx is an increasing number, that stops at 751 try wrappers hint above.

If that doesn’t help stop your client, remove peers.dat an start the client again. This forces the client to find new peers.

In general stopping and starting the client when the sync stalls seems to help.

The problem should be solved in one of the next versions.

wrapper

The problem was a temporary memory setting in Bitcoin was too low. When it was first set, it was not known that orphan chains could be longer than 750. The updated version fixes that problem so there is not a stall during sync.

Unfortunately, during the sync the client must check every block “chain”

If you do get the stalling problem, until 0.9.6.2 is released, stopping and restarting reloads the blocks the wallet is checking into the memory, so it can find the next block.

Sorry for the frustration, one solution, if you urgently need access to the wallet, would be to compile our 0.9.6.2 head, you can go back to the binary after sync.

gooders

@wrapper said in Wallet sync is very slow:

The problem was a temporary memory setting in Bitcoin was too low. When it was first set, it was not known that orphan chains could be longer than 750. The updated version fixes that problem so there is not a stall during sync.

Unfortunately, during the sync the client must check every block “chain”

If you do get the stalling problem, until 0.9.6.2 is released, stopping and restarting reloads the blocks the wallet is checking into the memory, so it can find the next block.

Sorry for the frustration, one solution, if you urgently need access to the wallet, would be to compile our 0.9.6.2 head, you can go back to the binary after sync.

Is there a release date for 0.9.6.2? i’m currently stuck with this issue and suggestions above are not fixing the problem :(

Nothing i do will allow it to go past 751…

wrapper

git clone

Build instructions :

@gooders heres my build script :

cp -f ~/Feathercoin/src/feathercoind  ~/Feathercoin/src/build/feathercoind 
cp -f ~/Feathercoin/src/feathercoin-cli  ~/Feathercoin/src/build/feathercoin-cli
cp -f ~/Feathercoin/src/qt/feathercoin-qt  ~/Feathercoin/src/build/feathercoin-qt
echo back-up bin files, copied ....
rm build.txt
time make clean
echo  autogen ........
time ./autogen.sh
./configure --with-gui=qt5 --enable-tests=no  --enable-upnp-default --with-qrcode=yes --enable-wallet=yes  --disable-hardening LDFLAGS="-L${BDB_PREFIX}/lib/" CPPFLAGS="-I${BDB_PREFIX}/include/" CXXFLAGS="$CXXFLAGS -fPIC -m64 -std=c++11"
time make
echo  finished ......
date
cp -f ~/Feathercoin/src/feathercoind  ~/Feathercoin/src/build/feathercoind 
cp -f ~/Feathercoin/src/feathercoin-cli  ~/Feathercoin/src/build/feathercoin-cli