can't find uninstalled packages on a system installed with the alternate image

one file requested by joern

"lubuntu-software-center-build-db /var/cache/lsc_packages.db
                 /usr/share/app-install/desktop/ /usr/share/LSC/categories.ini"
gives

  File "/usr/sbin/lubuntu-software-center-build-db", line 50, in <module>
     tags_file = open(sys.argv[3], "r"
 IOError: [Errno 21] Is a directory: '/usr/share/app-install/desktop/'
as requested by joern in irc.

Status changed to 'Confirmed' because the bug affects multiple users.

Good, now we have a separate bug report.

Let us know if you need additional information/data to help squash this bug!

Nio, can you confirm the workaround?

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1446830

I installed [Ubuntu] Software Center:

sudo apt-get install software-center

It works and can install for example htop without any issues.

Synaptic and apt-get work too.

So I confirm several work-arounds, that can be mentioned in the release notes (if the bug will not be squashed in time).

I got curious and tried Lubuntu Software Center again after installing [Ubuntu] Software Center. And now it works.

Was this what you meant? That installing [Ubuntu] Software Center makes Lubuntu Software Center work!

Anyway it does, but [Ubuntu] Software Center is good, so we might as well use it when it is installed.

-o-

I removed/purged [Ubuntu] Software Center

sudo apt-get purge software-center

and Lubuntu Software Center continues to work. So obviously the purging was not complete. Some file is left in a good state for the benefit of Lubuntu Software Center.

Please include any other workarounds in the appropriate section in the bug description above.

Installing with Synaptic and apt-get works as intended, but will not repair Lubuntu Software Center.

What do you mean by work-around? - To replace Lubuntu Software Center or make it work again?

I found another issue with Lubuntu Software Center:

Now it can find some but not all possible candidates for installation. The following packages are found and can be installed with Synaptic and apt-get, but not Lubuntu Software Center:

dialog, zenity (zenity is installed by default, yet not found)

while for example gimp is found (htop, iotop, arandr, xournal (CLI and GUI programs) were found and installed earlier).

All these programs are found by a re-installed [Ubuntu] Software Center.

Is there some way to force Lubuntu Software Center to update its data-base?

On Tue, 21 Apr 2015 22:06:53 -0000
sudodus <email address hidden> wrote:

> GUI programs) were found and installed earlier).
Nio lubuntu-software-center doesn't show all packages by defualt but try again once you open the prefrences and turn expert mode on.
--
Brendan Perrine <email address hidden>

Has anyone tried reinstalling lubuntu-software-center before bothering with installing software-center?

Note there's a DEBUG line when run under the command line that says "__init__ 44 Opening config file" and when searching "run 92 sensitive!" but these show up regardless of whether or not it works.

sudo dpkg-reconfigure lubuntu-software-center seems to cause it to see a package that is not installed and finds stuff.

Given all this informations, it seems that the DB is generated too early (when APT hasn't updated it cache yet) or without internet connection at all. The desktop isos bring some cache files with them, so there is no issue.

I would even bet, this issue is present on old alternates, too.

Changing the download server does updates the packages cache but doesn't show the software in the alternate installation.

Changing the download server does not regenerate the DB (and it should not even do it, because it takes too long imho).

Jörn, isn't the command to update the database `lubuntu-software-center-build-db /var/cache/lsc_packages.db /usr/share/app-install/desktop/ /usr/share/LSC/categories.ini`, as specified in the post-installation instructions? What's the fix for this? `apt-get update` in the post-install???

Since we have workarounds, I'm demoting this to high.

…or perhaps, is this an issue with debian-installer since it only affects alternate?

No, it is clearly a design flaw. While the DB is being generated, it drops every package without installation candidate. On the alternate install, there are no informations about the avaible packages so everything is dropped, if it isn't installed. That is why DB is that small, it should be around 80 megs (see #2).

Running apt-get update as postinst would have no effect if no internet connection is avaible (not sure if it works with internet). To be honest, I don't see a good fix for this. We could avoid dropping of entries without installation candidate, but that would mean that uninstallable packages appear in LSC.

Sigh, another reason I don't love the alternate image.

I think it would be wise to include all packages in the database and then check for installation candidates at install. Couldn't that happen?

What do the other software centers do?

Shipping the database one the iso means 80 MB more, not sure how good the compression ratio would be. That is a no-go I think. Especially for the desktop iso.

As the alternate iso isn't used often, I guess we can live with this issue. Sad, but I don't see a good solution.

Could we add a conditional to check to see if we're using debian-installer and if so, just try to update before building the database? Worst case scenario the update fails because there's no internet connection. Oh well. Better than nothing.

I did some investigation, apt-get update in postinst doesn't work at all, because APT is locked.

wow, sounds like this might a won't fix huh?

Confirmed this is a problem as far back as Precise. Wow, I'm surprised we never noticed this before. Maybe it's a good time to check in with the developers of the other software centers and see what they do? Or work on a major redesign? Either way, I'm thinking we target this for W-series. Thoughts, Jörn?

Ubuntu's Software Center is totally different. It doesn't have a DB for caching like LSC has.

Some questions:

1. What is the advantage of Lubuntu Software Center compared to [Ubuntu] Software Center? Is it lighter or faster?

2. How many people are using it? Maybe very few people because this bug has not been detected since Precise.

3. Would it be a good idea to remove it from Lubuntu are get a better margin to keep the iso files within CD size? It could stay in the repos to be installed (with for example Synaptic) for people who want to.

Corrected typing error: ... remove it from Lubuntu *in order to* get a better margin ...

What about Muon (Kubuntu)? I don't know if Kylin's Software Center is any different than Synaptic, but that might be something to look at, too.

Why not re-design this without using the DB? It does seem kind of kludgy.

We also can't do dpkg-reconfigure in postinst, right?

Confirmed the lubuntu-software-center-update-db fix actually does do the trick. It is in postinst but clearly locked apt makes for a problem. Is there a postpostinst trigger? XD

Hi Nio,

1. it is way faster, really a big difference. USC was made a bit faster, but it's still not comparable to LSC.
2. I guess most Lubuntu users are using it, but alternate iso installs are quite rare. And as the DB is regenerated from time to time, nobody noticed that issue.
3. It's working fine mostly and doesn't take much space, so I think we should keep it. Remember, LSC was the thing how I came to Lubuntu ;-)

@Walter: we could redesign it, yes. It could use Appstream data, like the future Lubuntu/LXQt software center will. But it would be really a lot of work and we don't have the manpower, I think. I think "Won't fix" is the only solution in our position. Would only be worth the effort if other flavours would like to adopt LSC - Ubuntu GNOME wants it, but only as an interim solution. Xubuntu doesn't want it, so I think it will be abandoned (more or less) soon.

So there are good reasons to keep LSC, even in the won't fix scenario :-)

How is the regeneration of the database triggered? Can it be triggered at the first boot (not every boot) of a system installed by the alternate installer? I'm thinking of that as a possible bug-fix: wait for the internet connection and then regenerate the database.

Not sure what all the triggers are, I can only find updates for app-install-data as a trigger - but they are more often.

Since Synaptic is fully functional in the alternate could we not live with that for this version?

Maybe a file can be used as a flag. a file provided when installing via the alternate installer. When the LSC is started, check for that file and check for internet. If both conditions are satisfied, regenerate the database. If the database was generated successfully, remove the file that is used as a flag. That might work and be independent of other triggers in the system.

But is there a way to do this? I don't know much about the alternate installer.

I'm not sure there is time to do it today for 15.04, but I think it would be possible given more time. Such a 'flag file' and the logic to use it can be included in the LSC package, and it would start working when the installed system is upgraded with the bug-fixed LSC package.

If someone finds a way to create that file with the alternate iso, I'm willing to sponsor a patch for LSC.

There's no reason we couldn't include this in updates. If you seriously have a plan to make this happen, Nio, please assign this to yourself. Otherwise, I'm going to call this Won't Fix.

OK, let us call it "won't fix" now.

I would suggest that Jörn and I discuss the issue privately and maybe agree about the details and after that ask Julien (gilir) to help us add such a file when creating the alternate iso file. I guess there is a set of scripts (similar to (but more advanced than) what I use to create 9w), and if that is the case, it is straightforward to create a file (which can be empty, just have a specified path/name). Jörn and I might also find a better solution, that needs nothing beyond what can be included in he LSC package.

Hi,
Nio e-mailed me about this recently.
I have one idea, that might work.

a postintpostint script is being described... so the only real hacky workaround I can think of is to create a script.

This script can check for internet and wait until apt is finished(or wait 5 minutes). Then it can update the database.

install the script, and then run it from postinst. Once it is run it can delete itself, or flag something for postinst to not run it ever again...
postinst could say something like

if [ -f "$flag_file_found" ]
then
#do nothing
else
fix_things_with_the_hacktastic_script
fi

I think this would be the easiest way to work around this issue, but definitely not as good as fixing the problem.

also... you could put a check in the script to look at the size of the database.
It should be possible to check any number of things from this script.

I would be willing to help someone if they want to work on this. But I have a lot going on, so I cannot take full ownership of this bug.