Ticket #2131 (closed defect: fixed)

Opened 9 years ago

Last modified 3 months ago

"opkg upgrade" slow, duplicate entries in *.list files

Reported by: h.koenig Owned by: tick
Priority: normal Milestone:
Component: opkg Version:
Severity: normal Keywords:
Cc: Blocked By:
Blocking: Estimated Completion (week):
HasPatchForReview: no PatchReviewResult:
Reproducible:

Description

root@om-gta02:~# opkg list_installed opkg
opkg - 0.1.4+svnr4640-r0.1 -

root@om-gta02:~# time opkg upgrade
Package ncurses-terminfo is already installed in root.

real 1m9.038s
user 0m56.890s
sys 0m7.390s

(this was much worse (2.5 to 3 minutes) before removing ncurses-terminfo.list -- see below!)

questions/bugs:

  • why do *.list files grow with many duplicate entries ?
  • why does this cause reading gtk+-fastscaling.list many times ?
  • what's the problem with ncurses-terminfo ?

BTW, this was an OM2008.8 installation with all updates, now updated to testing.

with strace I noticed that one single pkg file list got opened 12294 times. once this was usr/lib/opkg/info/ncurses-terminfo.list which then I moved away and opkg upgrade was fast again. right now it's usr/lib/opkg/info/gtk+.list -- from strace output:

root@om-gta02:~# sort opkd.strace.open1 | uniq -c | sort -n | tail -2

1 syscall_983045(0x40305a70, 0x40305a70, 0x68c, 0x40306148, 0x40024060, 0, 0x60, 0xf0005, 0xffff566c, 0x608, 0x40024000, 0xbedcbaec, 0, 0xbedcb7c0, 0x40001fa8, 0x40001fc0, 0x20000010, 0x40305a70, 0, 0, 0, 0xd708, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = 0

12294 open("usr/lib/opkg/info/gtk+.list", O_RDONLY) = 4

why does opkg (sometimes) open a pkg list so many times ? I kept this files in case someone wants to inspect them.

unfortuneately this time (re)moving /usr/lib/opkg/info/gtk+.list does not help to get opkg upgrade working fast again :-(

right now opkg upgrade runs 20 secs after opening usr/lib/opkg/info/gtk+-fastscaling.list and another 20 secs after usr/lib/opkg/info/xtscal.list (from strace -rtt -e open ...)

xtscal.list is the last list being read, so that's the "real" opkg work. (re)moving gtk+-fastscaling.list saves 20 secs of runtime!

checking gtk+-fastscaling.list I noticed that it's very long and some times are duplicates _many_ times:

sort gtk+-fastscaling.list | uniq -c | grep -v " 1 "

1024 /usr/bin/gdk-pixbuf-query-loaders

11264 /usr/lib/gtk-2.0/2.10.0/engines/libpixmap.so

11 /usr/lib/libgdk_pixbuf_xlib-2.0.so.0

so I "cleaned up" all *.list files with "sort -u" reducing from 29816 lines in all *.list files to 10371 lines!!

now ncurses-terminfo.list still got read 2424 times, and gtk+.list 8 times , running for ~2.5 minutes! without ncurses-terminfo.list it runs only 28 secs, gtk+.list does not matter in run time.

so removing the duplicate file names i *.list files reduced runtime at least from 60+ to 30- seconds.

I'd guess that the message

Package ncurses-terminfo is already installed in root.

is somehow related with the fact that ncurses-terminfo.list still is read 2k times taking ~2 minutes. the reason for reading gtk+-fastscaling.list many times seemed to be fixed by cleaning up *.list files...

Change History

comment:1 Changed 9 years ago by tick

  • Status changed from new to accepted
  • Owner changed from openmoko-devel to tick

why do *.list files grow with many duplicate entries ?

  • It's a bug in the hash_table.c hast_table did not check key hit after second entry. This had been solved at #4830.

why does opkg (sometimes) open a pkg list so many times ?

  • It's a issue of the reference count scheme. I am planing to make it more lazy, and reduce the reading times. Still working on it.

Another issue that cause a file been written for many times is solved at #4829. (Set the flag back.)

Thank you for reporting this.

comment:2 Changed 9 years ago by tick

  • Component changed from unknown to opkg

comment:3 Changed 9 years ago by tick

  • Status changed from accepted to closed
  • Resolution set to fixed

solved at svn r4830

comment:4 Changed 3 months ago by JustDox

Levitra Westen <a href=http://cial1.xyz/generic-cialis.php>Generic Cialis</a> Zithromax Vs Levaquin Secure Finasteride Best Buy Fedex <a href=http://zol1.xyz/getting-zoloft-overseas.php>Getting Zoloft Overseas</a> Synthyroid Without A Prescription Calf Pain And Cephalexin <a href=http://kama1.xyz/buy-kamagra-oral-jelly-online.php>Buy Kamagra Oral Jelly Online</a> Viagra Effets Indesirables Amoxil 20mg Duree <a href=http://viag1.xyz/cheap-viagra-pills.php>Cheap Viagra Pills</a> Vardenafil Warnings Cephalexin Ibuprofen <a href=http://nolvadex.ccrpdc.com/order-nolvadex.php>Order Nolvadex</a> Propecia With Minoxidil Nisim Viagra Drug Interactions <a href=http://cial1.xyz/cheap-cialis-generic.php>Cheap Cialis Generic</a> Kamagra Directions Of Use Generic Viagra No Prescription <a href=http://cial1.xyz/generic-cialis-usa.php>Generic Cialis Usa</a> Cialis 20 Mg Comprar 1000 Mg Zithromax <a href=http://viag1.xyz/viagra-sample.php>Viagra Sample</a> Genericpillsshop Pham Suport Group <a href=http://cial1.xyz/cialis-online-prices.php>Cialis Online Prices</a> Levitra Generic 10mg Cialis Nachteile <a href=http://viag1.xyz>Buy Viagra</a> Propecia Online Cheapest Erfahrungen Mit Viagra Nebenwirkungen <a href=http://kama1.xyz/generic-kamagra-online.php>Generic Kamagra Online</a> Viagra Generika Ohne Rezept Generic Cialis Reviews <a href=http://viagra.ccrpdc.com/viagra-for-sale.php>Viagra For Sale</a> Blood Pressure Medications For Sale Kamagra Fast <a href=http://viag1.xyz/cheap-viagra-fast.php>Cheap Viagra Fast</a> Cialis Dangereux Propecia Storage <a href=http://cial1.xyz/cialis-on-line.php>Cialis On Line</a> 104 Vigra Cialis Achat En Ligne Canada <a href=http://zol1.xyz/buy-generic-zoloft-online.php>Buy Generic Zoloft Online</a> Where To Buy Tadalis Sx Soft Online Neem <a href=http://zol1.xyz/zoloft-online-usa.php>Zoloft Online Usa</a> High Dose Of Amoxicillin Kamagra Prix Moyen <a href=http://strattera.ccrpdc.com/buy-generic-strattera.php>Buy Generic Strattera</a> Buy Tamoxifen Online No Prescription Cephalexin Brand Name <a href=http://cial5mg.xyz/purchase-cheap-cialis.php>Purchase Cheap Cialis</a> Il Cialis

Note: See TracTickets for help on using tickets.