Packaging BLAT for Debian

Thanks for packaging it up, and respecting the license.

You can get pslCheck in executable form for several OS's at
http://hgdownload.cse.ucsc.edu/admin/exe
The source sadly is still not isolated from the huge UCSC kent source tree.
It is pretty fast to build at least though, and once you have MACHTYPE set
up easy to make. Please see
http://hgdownload.cse.ucsc.edu/downloads.html#source_downloads
to get the whole source tree.

Post by Andreas Tille
Hi Jim,
I'm writing you on behalf of the Debian Med team that has the goal to
introduce official Debian packages of Free Software which is relevant
in medicine and biology. Considering that you are the author of BLAT
you might be interested in our biology task here
http://blends.debian.org/med/tasks/bio
to get an overview what we are working on.
You might have noticed that also your software BLAT is mentioned on
this page
http://blends.debian.org/med/tasks/bio#blat
on out todo list. Please note that it needs to go in Debian's non-free
section since it is restricted to scientific use only.
The problem I currently have is that I try to run any test suite of
software if it is available. So I also tried
cd blat/test; make
which quickly ends up in
blat -verbose=0 throwback/target1.fa throwback/query1.fa throwback/test.psl
Loaded 129433 letters in 1 sequences
Searched 2050 bases in 1 sequences
pslCheck -verbose=0 throwback/test.psl
make: pslCheck: Command not found
make: *** [tThrowback] Error 127
since there is no binary pslCheck. Did I missed something when creating
the binaries from BLAT source since I only found a function
int pslCheck(char *pslDesc, FILE* out, struct psl* psl)
in file lib/psl.c but no such executable to call. Any hint how to
run the test suite successfully?
Kind regards and thanks for providing BLAT
Andreas.
--
http://fam-tille.de

Andreas Tille

2014-02-25 20:12:22 UTC

Hi Jim,

thanks for your quick and helpful response.

Post by Jim Kent
Thanks for packaging it up, and respecting the license.

Sure Debian as a distribution will respect the license. It is also
quite verbosely included in any package. However, for sure we have no
influence whether any *user* will respect the license.

Post by Jim Kent
You can get pslCheck in executable form for several OS's at
http://hgdownload.cse.ucsc.edu/admin/exe
The source sadly is still not isolated from the huge UCSC kent source tree.
It is pretty fast to build at least though, and once you have MACHTYPE set
up easy to make. Please see
http://hgdownload.cse.ucsc.edu/downloads.html#source_downloads
to get the whole source tree.

Ahhh, I started fighting through the list of dependencies but it seems
to be complex (so yes, I might understand your reasons not to inject
it into the BLAT source). However, *if* we want to run the test inside
Debian we would really need the source since we can not distribute
binaries. Ignoring this for a moment I simply used the binary to find
out whether the test would run at all.

cd blat/test; make tThrowback
blat -verbose=0 throwback/target1.fa throwback/query1.fa throwback/test.psl
Loaded 129433 letters in 1 sequences
Searched 2050 bases in 1 sequences
pslCheck -verbose=0 throwback/test.psl
blat -verbose=0 v29skips/ex1_database.fa v29skips/ex1_query.fa v29skips/ex1.psl
Loaded 4481 letters in 1 sequences
Searched 4484 bases in 1 sequences
diff v29skips/ex1_reference.psl v29skips/ex1.psl
6c6
< 4478 3 0 0 1 3 0 0 + genomicMrnaSCA2 4484 0 4484 FnaSCA2 4481 0 4481 2 699,3782, 0,702, 0,699,
---

Post by Jim Kent
3782 0 0 0 0 0 0 0 + genomicMrnaSCA2 4484 702 4484 FnaSCA2 4481 699 4481 1 3782, 702, 699,

make: *** [tThrowback] Error 1

Trying the second check is not working any better:

$ make tIntronMax
mkdir -p intron50k/out
blat -verbose=0 intron50k/target.fa intron50k/query.fa intron50k/out/test1.psl -minScore=190
Loaded 100000 letters in 1 sequences
Searched 600 bases in 1 sequences
diff intron50k/expected/test1.psl intron50k/out/test1.psl
6c6,8
< 600 0 0 0 0 0 2 60200 + query 600 0 600 chr16part 100000 10000 70800 3 200,199,201, 0,200,399, 10000,60200,70599,
---

Post by Jim Kent
542 9 0 0 0 0 3 67695 + query 600 0 551 chr16part 100000 10000 78246 4 200,199,81,71, 0,200,399,480, 10000,60200,70599,78175,
202 0 0 0 0 0 0 0 + query 600 200 402 chr16part 100000 60200 60402 1 202, 200, 60200,
201 0 0 0 0 0 0 0 + query 600 399 600 chr16part 100000 70599 70800 1 201, 399, 70599,

make: *** [tIntronMax] Error 1

I wonder whether you are aware about circumstances when these both tests
are failing and what this might mean for the blat executable I created
on a Debian machine on amd64 architecture (using MACHTYPE="x86_64").

Does this mean that my build was faulty or is there any other problem?

Kind regards

Andreas.

--
http://fam-tille.de

Jim Kent

2014-02-26 08:40:56 UTC

No problem about the users not necessarily respecting the license. We
already have the source code up on the net with no protection. We rely on
the honor system, and the business remains good enough at least to pay the
mortgage.

I think there may be a problem with your build. I've got a _little_ time
to help you with this, but if I get too busy to answer all your email
please don't take it personally. Here's the output I get from the make
test in our main source tree.

blat -verbose=0 hCrea.geno hCrea.mrna testRna.psl

Loaded 6896 letters in 1 sequences

Searched 1540 bases in 1 sequences

cmp testRna.psl refRna.psl

blat -verbose=0 -prot hCrea.pep mCrea.pep testProt.psl

Loaded 417 letters in 1 sequences

Searched 418 bases in 1 sequences

cmp testProt.psl refProt.psl

blat -verbose=0 -t=dnax -q=prot hCrea.geno mCrea.pep testProtX.psl

Loaded 6896 letters in 1 sequences

Blatx 1 sequences in database, 1 files in query

cmp testProtX.psl refProtX.psl

blat -verbose=0 -t=dnax -q=rnax hCrea.geno mCrea.mrna testRnaX.psl

Loaded 6896 letters in 1 sequences

Blatx 1 sequences in database, 1 files in query

cmp testRnaX.psl refRnaX.psl

blat -verbose=0 -fine hCrea.geno hCrea.mrna testFine.psl

Loaded 6896 letters in 1 sequences

Searched 1540 bases in 1 sequences

cmp testFine.psl refFine.psl

cd test && make

make[1]: Entering directory `/cluster/home/kent/kent/src/blat/test'

blat -verbose=0 throwback/target1.fa throwback/query1.fa throwback/test.psl

Loaded 129433 letters in 1 sequences

Searched 2050 bases in 1 sequences

pslCheck -verbose=0 throwback/test.psl

blat -verbose=0 v29skips/ex1_database.fa v29skips/ex1_query.fa
v29skips/ex1.psl

Loaded 4481 letters in 1 sequences

Searched 4484 bases in 1 sequences

diff v29skips/ex1_reference.psl v29skips/ex1.psl

blat -verbose=0 v29skips/ex2_database.fa v29skips/ex2_query.fa
v29skips/ex2.psl

Loaded 4186 letters in 1 sequences

Searched 4185 bases in 1 sequences

diff v29skips/ex2_reference.psl v29skips/ex2.psl

mkdir -p intron50k/out

blat -verbose=0 intron50k/target.fa intron50k/query.fa
intron50k/out/test1.psl -minScore=190

Loaded 100000 letters in 1 sequences

Searched 600 bases in 1 sequences

diff intron50k/expected/test1.psl intron50k/out/test1.psl

blat -verbose=0 intron50k/target.fa intron50k/query.fa
intron50k/out/test2.psl -minScore=190 -maxIntron=40000

Loaded 100000 letters in 1 sequences

Searched 600 bases in 1 sequences

diff intron50k/expected/test2.psl intron50k/out/test2.psl

blat -verbose=0 intron50k/target.fa intron50k/query.fa
intron50k/out/test3.psl -minScore=190 -maxIntron=5000

Loaded 100000 letters in 1 sequences

Searched 600 bases in 1 sequences

diff intron50k/expected/test3.psl intron50k/out/test3.psl

rm -rf intron50k/out

make[1]: Leaving directory `/cluster/home/kent/kent/src/blat/test'

Post by Andreas Tille
Hi Jim,
thanks for your quick and helpful response.

Post by Jim Kent
Thanks for packaging it up, and respecting the license.

Sure Debian as a distribution will respect the license. It is also
quite verbosely included in any package. However, for sure we have no
influence whether any *user* will respect the license.

Post by Jim Kent
You can get pslCheck in executable form for several OS's at
http://hgdownload.cse.ucsc.edu/admin/exe
The source sadly is still not isolated from the huge UCSC kent source

tree.

Post by Jim Kent
It is pretty fast to build at least though, and once you have MACHTYPE

set

Post by Jim Kent
up easy to make. Please see
http://hgdownload.cse.ucsc.edu/downloads.html#source_downloads
to get the whole source tree.

Ahhh, I started fighting through the list of dependencies but it seems
to be complex (so yes, I might understand your reasons not to inject
it into the BLAT source). However, *if* we want to run the test inside
Debian we would really need the source since we can not distribute
binaries. Ignoring this for a moment I simply used the binary to find
out whether the test would run at all.
cd blat/test; make tThrowback
blat -verbose=0 throwback/target1.fa throwback/query1.fa throwback/test.psl
Loaded 129433 letters in 1 sequences
Searched 2050 bases in 1 sequences
pslCheck -verbose=0 throwback/test.psl
blat -verbose=0 v29skips/ex1_database.fa v29skips/ex1_query.fa
v29skips/ex1.psl
Loaded 4481 letters in 1 sequences
Searched 4484 bases in 1 sequences
diff v29skips/ex1_reference.psl v29skips/ex1.psl
6c6
< 4478 3 0 0 1 3 0 0 +
genomicMrnaSCA2 4484 0 4484 FnaSCA2 4481 0 4481 2
699,3782, 0,702, 0,699,
---

Post by Jim Kent
3782 0 0 0 0 0 0 0 +

genomicMrnaSCA2 4484 702 4484 FnaSCA2 4481 699 4481 1
3782, 702, 699,
make: *** [tThrowback] Error 1
$ make tIntronMax
mkdir -p intron50k/out
blat -verbose=0 intron50k/target.fa intron50k/query.fa
intron50k/out/test1.psl -minScore=190
Loaded 100000 letters in 1 sequences
Searched 600 bases in 1 sequences
diff intron50k/expected/test1.psl intron50k/out/test1.psl
6c6,8
< 600 0 0 0 0 0 2 60200 +
query 600 0 600 chr16part 100000 10000 70800 3
200,199,201, 0,200,399, 10000,60200,70599,
---

Post by Jim Kent
542 9 0 0 0 0 3 67695 +

query 600 0 551 chr16part 100000 10000 78246 4
200,199,81,71, 0,200,399,480, 10000,60200,70599,78175,

Post by Jim Kent
202 0 0 0 0 0 0 0 +

query 600 200 402 chr16part 100000 60200 60402 1
202, 200, 60200,

Post by Jim Kent
201 0 0 0 0 0 0 0 +

query 600 399 600 chr16part 100000 70599 70800 1
201, 399, 70599,
make: *** [tIntronMax] Error 1
I wonder whether you are aware about circumstances when these both tests
are failing and what this might mean for the blat executable I created
on a Debian machine on amd64 architecture (using MACHTYPE="x86_64").
Does this mean that my build was faulty or is there any other problem?
Kind regards
Andreas.
--
http://fam-tille.de

Andreas Tille

2014-02-26 09:14:01 UTC

Hi Jim,

Post by Jim Kent
No problem about the users not necessarily respecting the license. We
already have the source code up on the net with no protection. We rely on
the honor system, and the business remains good enough at least to pay the
mortgage.

:-)

Post by Jim Kent
I think there may be a problem with your build. I've got a _little_ time
to help you with this, but if I get too busy to answer all your email
please don't take it personally.

I'm far away from taking this personally. Considering your past response
time I'm pretty sure that you might have more urgent things to do if we
need to wait a bit longer.

Post by Jim Kent
Here's the output I get from the make
test in our main source tree.
blat -verbose=0 hCrea.geno hCrea.mrna testRna.psl
...
diff intron50k/expected/test3.psl intron50k/out/test3.psl
rm -rf intron50k/out
make[1]: Leaving directory `/cluster/home/kent/kent/src/blat/test'

This looks like a perfect test result and we definitely need to reach
this state with the Debian package before this will be published. The
only vague idea I have is that some compiler options might lead to the
wrong result. I attached a build log for blat and I wonder whether you
might like to have a short look whether some of the options might look
suspiciouos to you. This could give some hint where to switch the lever
next time.

Thanks a lot for your kind cooperation

Andreas.

--
http://fam-tille.de

Andreas Tille

2014-02-26 12:09:14 UTC

Hi again,

after droping the -O2 compiler option I was able to run

$ cd blat/test ; make
blat -verbose=0 throwback/target1.fa throwback/query1.fa throwback/test.psl
Loaded 129433 letters in 1 sequences
Searched 2050 bases in 1 sequences
pslCheck -verbose=0 throwback/test.psl
blat -verbose=0 v29skips/ex1_database.fa v29skips/ex1_query.fa v29skips/ex1.psl
Loaded 4481 letters in 1 sequences
Searched 4484 bases in 1 sequences
diff v29skips/ex1_reference.psl v29skips/ex1.psl
blat -verbose=0 v29skips/ex2_database.fa v29skips/ex2_query.fa v29skips/ex2.psl
Loaded 4186 letters in 1 sequences
Searched 4185 bases in 1 sequences
diff v29skips/ex2_reference.psl v29skips/ex2.psl
mkdir -p intron50k/out
blat -verbose=0 intron50k/target.fa intron50k/query.fa intron50k/out/test1.psl -minScore=190
Loaded 100000 letters in 1 sequences
Searched 600 bases in 1 sequences
diff intron50k/expected/test1.psl intron50k/out/test1.psl
blat -verbose=0 intron50k/target.fa intron50k/query.fa intron50k/out/test2.psl -minScore=190 -maxIntron=40000
Loaded 100000 letters in 1 sequences
Searched 600 bases in 1 sequences
diff intron50k/expected/test2.psl intron50k/out/test2.psl
blat -verbose=0 intron50k/target.fa intron50k/query.fa intron50k/out/test3.psl -minScore=190 -maxIntron=5000
Loaded 100000 letters in 1 sequences
Searched 600 bases in 1 sequences
diff intron50k/expected/test3.psl intron50k/out/test3.psl
rm -rf intron50k/out

So probably -O2 is causing the trouble I faced running the test.

Now since I figured out this I need to decide how I reaonably get
pslCheck included as source to be able to run the full test. I realised
that several other parts of jksrc are involved. Any hint?

Kind regards

Andreas.

Post by Andreas Tille
Hi Jim,

:-)

Post by Jim Kent
I think there may be a problem with your build. I've got a _little_ time
to help you with this, but if I get too busy to answer all your email
please don't take it personally.

I'm far away from taking this personally. Considering your past response
time I'm pretty sure that you might have more urgent things to do if we
need to wait a bit longer.

--
http://fam-tille.de

Andreas Tille

2014-02-27 07:16:41 UTC

Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the whole
1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the pslCheck
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source tree for
Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to build.

This is a very cool offer. I actually have thought about packaging the
whole UCSC genomics source tree as well since it obviously contains
several tools that perfectly fit in our scope. I wonder whether Hiram
might even like to learn something about Debian packaging. In our team
we have quite some tradition in mentoring people as you can see here:

https://wiki.debian.org/DebianMed/MoM

Perhaps it comes handy if somebody in your team is capable to create
Debian packages which in the end is not more than wrapping up the build
process into some sceme.

BTW, when I inspected the jksrc source tree (and also in the specific
case of the blat source) I realised that it might make real sense to
enable dynamic linking of the tools against the static libraries you are
creating. The Debian way to do this would be to create two packages:

lib<name> containing the dynamic libraries
lib<name>-dev containing the static libraries and header files

To approach this easily it is quite convenient to use either GNU
automake or cmake (at your preference) since these build systems easily
support the creation of dynamic and static libraries in parallel. This
would also simplify the hancling of MACHTYPE in your makefiles since
these Build systems are capable to handle this automatically. In short:
before we might start packaging the whole source tree it would be quite
sensible to switch to an advanced build system which would be also in
your profit at the end.

- a small part which is owned by me in a directory called jkOwnLib, and in
the blat directories

This would probably make a separate library package. However, you might
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally useful
which is essentially public domain, but that we are happy distributed under
a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser in
particular specific that is owned by UCSC and has a license much like blat
- free for personal, academic, and non-profit use, and requiring a license
for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi Brumbaugh).

In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.

Kind regards

Andreas.

--
http://fam-tille.de

Andreas Tille

2014-02-27 08:31:35 UTC

Hi Jim,

seems like we got in contact to the perfectly right time. :-)

I'm happily waiting for your colleagues showing up at the Debian Med
list (which I'd strongly recommend for this purpose since I'm just a
member of the team and not the whole team ;-)) and repeat that one of
them might be interested in our Mentoring of the Month effort

https://wiki.debian.org/DebianMed/MoM

Looking forward to a great cooperation

Andreas.

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and also
Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to use
them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was state
of the art in 1985, but not the
config
make
make install
people are used to these days, never mind a RPM or anything more recent,
and most of the younger programmers get lost.
I do think we want to do some renaming of directories and the like as part
of this process, and ideally end up with all the code that is under one
license under the same subdirectory. It's somewhat close to that, but
there are enough exceptions to be a pain. We switched from CVS to git
about 2 years ago in large part to make moving directories around much less
of a pain in the butt, so we _can_ do this now, but it's been sort of a
back burner thing, and is only about 10% complete.
Anyway, we are paid by the taxpayers to do this sort of work, and will
make some time for it. We would welcome your help, and getting it into
Debian is as good a starting point as any, better than most if we have
support from that group.
Take care,
Jim

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the whole
1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the pslCheck
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source tree

for

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to build.

This is a very cool offer. I actually have thought about packaging the
whole UCSC genomics source tree as well since it obviously contains
several tools that perfectly fit in our scope. I wonder whether Hiram
might even like to learn something about Debian packaging. In our team
https://wiki.debian.org/DebianMed/MoM
Perhaps it comes handy if somebody in your team is capable to create
Debian packages which in the end is not more than wrapping up the build
process into some sceme.
BTW, when I inspected the jksrc source tree (and also in the specific
case of the blat source) I realised that it might make real sense to
enable dynamic linking of the tools against the static libraries you are
lib<name> containing the dynamic libraries
lib<name>-dev containing the static libraries and header files
To approach this easily it is quite convenient to use either GNU
automake or cmake (at your preference) since these build systems easily
support the creation of dynamic and static libraries in parallel. This
would also simplify the hancling of MACHTYPE in your makefiles since
before we might start packaging the whole source tree it would be quite
sensible to switch to an advanced build system which would be also in
your profit at the end.

- a small part which is owned by me in a directory called jkOwnLib, and

the blat directories

This would probably make a separate library package. However, you might
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally useful
which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser in
particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Jim Kent

2014-02-27 08:49:06 UTC

Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Post by Andreas Tille
Hi Jim,
seems like we got in contact to the perfectly right time. :-)
I'm happily waiting for your colleagues showing up at the Debian Med
list (which I'd strongly recommend for this purpose since I'm just a
member of the team and not the whole team ;-)) and repeat that one of
them might be interested in our Mentoring of the Month effort
https://wiki.debian.org/DebianMed/MoM
Looking forward to a great cooperation
Andreas.

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

of the art in 1985, but not the
config
make
make install
people are used to these days, never mind a RPM or anything more recent,
and most of the younger programmers get lost.
I do think we want to do some renaming of directories and the like as

part

of this process, and ideally end up with all the code that is under one
license under the same subdirectory. It's somewhat close to that, but
there are enough exceptions to be a pain. We switched from CVS to git
about 2 years ago in large part to make moving directories around much

less

of a pain in the butt, so we _can_ do this now, but it's been sort of a
back burner thing, and is only about 10% complete.
Anyway, we are paid by the taxpayers to do this sort of work, and will
make some time for it. We would welcome your help, and getting it into
Debian is as good a starting point as any, better than most if we have
support from that group.
Take care,
Jim

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

Post by Andreas Tille
This is a very cool offer. I actually have thought about packaging the
whole UCSC genomics source tree as well since it obviously contains
several tools that perfectly fit in our scope. I wonder whether Hiram
might even like to learn something about Debian packaging. In our team
https://wiki.debian.org/DebianMed/MoM
Perhaps it comes handy if somebody in your team is capable to create
Debian packages which in the end is not more than wrapping up the build
process into some sceme.
BTW, when I inspected the jksrc source tree (and also in the specific
case of the blat source) I realised that it might make real sense to
enable dynamic linking of the tools against the static libraries you

are

Post by Andreas Tille
lib<name> containing the dynamic libraries
lib<name>-dev containing the static libraries and header files
To approach this easily it is quite convenient to use either GNU
automake or cmake (at your preference) since these build systems easily
support the creation of dynamic and static libraries in parallel. This
would also simplify the hancling of MACHTYPE in your makefiles since
these Build systems are capable to handle this automatically. In
before we might start packaging the whole source tree it would be quite
sensible to switch to an advanced build system which would be also in
your profit at the end.

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Andreas Tille

2016-09-10 21:36:38 UTC

Hi again,

I'd like to pick up the ball for blat again. For me it is not yet clear
by what license blat is covered and what might be the role of the KentLib
library[1] plays. Is it possible to link blat against KentLib and is it
sensible to start packaging this first.

Just to let you know: The freeze for the next Debian release is coming
soon and it blat should be distributed with the next release we should
hurry up to get this done. For me it remains unclear who is responsible
for what and role the different pieces of code are playing.

Kind regards

Andreas.

[1] https://github.com/jstjohn/KentLib

Post by Jim Kent
Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

part

less

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

are

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Andreas Tille

2017-10-25 10:20:06 UTC

Hi again,

I'm stumbling upon some definitive answer about the license of blatSrc
which can be found in several projects. My last hit was seqtools[1]
from Sanger which claims to be licensed as GPL-3 but as far as I can see
this would conflict with the licensing statement of the latest blatSrc
download I'm aware about[2]. So it would really help to get some
official statement under what license blatSrc can be used and where
to find the latest version.

Thanks a lot

Andreas.

[1] http://www.sanger.ac.uk/science/tools/seqtools
[2] https://users.soe.ucsc.edu/~kent/src/blatSrc35.zip

Post by Andreas Tille
Hi again,
I'd like to pick up the ball for blat again. For me it is not yet clear
by what license blat is covered and what might be the role of the KentLib
library[1] plays. Is it possible to link blat against KentLib and is it
sensible to start packaging this first.
Just to let you know: The freeze for the next Debian release is coming
soon and it blat should be distributed with the next release we should
hurry up to get this done. For me it remains unclear who is responsible
for what and role the different pieces of code are playing.
Kind regards
Andreas.
[1] https://github.com/jstjohn/KentLib

Post by Jim Kent
Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

part

less

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

are

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Jim Kent

2017-10-25 18:44:02 UTC

Hi - Blat does have it's own unique license. The essence of it is
that it is free for personal, academic, and non-profit use, and
requires a fee paying license from Kent Informatics for commercial
use. I spent some time trying to find a standard license that fit a
year or two ago, and then apparently it wasn't standard enough. I'm
kind of reluctant to go through that exercise again. Is there no way
you can use our very simple license for non-commercial users?

Post by Andreas Tille
Hi again,
I'm stumbling upon some definitive answer about the license of blatSrc
which can be found in several projects. My last hit was seqtools[1]
from Sanger which claims to be licensed as GPL-3 but as far as I can see
this would conflict with the licensing statement of the latest blatSrc
download I'm aware about[2]. So it would really help to get some
official statement under what license blatSrc can be used and where
to find the latest version.
Thanks a lot
Andreas.
[1] http://www.sanger.ac.uk/science/tools/seqtools
[2] https://users.soe.ucsc.edu/~kent/src/blatSrc35.zip

Post by Jim Kent
Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

part

less

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

are

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Andreas Tille

2017-10-25 20:06:33 UTC

Hi Kent,

thanks for your quick reply.

Post by Jim Kent
Hi - Blat does have it's own unique license. The essence of it is
that it is free for personal, academic, and non-profit use, and
requires a fee paying license from Kent Informatics for commercial
use. I spent some time trying to find a standard license that fit a
year or two ago, and then apparently it wasn't standard enough. I'm
kind of reluctant to go through that exercise again. Is there no way
you can use our very simple license for non-commercial users?

I think I explained in the past in detail that *any* restriction for
*any* user makes the code non-free and can not distributed with Debian
(and in the same manner not with seqtools which is GPL-3). I had the
impression that you was willing to drop the non-profit use restriction
but it seems I was wrong. The reason that you did not found a standard
Open Source license is that your restriction is in conflict per
definition with Open Source licenses.

I had this discussion with Joe Felsenstein for years and he finally
realised that the non-commercial restriction has more drawbacks (like
beeing not included in any Linux distribution) than advantages (really
low payment from commercial users ... are you sure that commercial
users really pay for a license in their closed products?)

This idea has distributed quite widely in scientific software world
and may be you reconsider your decision.

Kind regards

Andreas.

Post by Jim Kent
Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

part

less

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

are

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Hiram Clawson

2017-10-25 20:12:56 UTC

Andreas: FYI: https://genome-store.ucsc.edu/

Post by Andreas Tille
Hi Kent,
thanks for your quick reply.

I think I explained in the past in detail that *any* restriction for
*any* user makes the code non-free and can not distributed with Debian
(and in the same manner not with seqtools which is GPL-3). I had the
impression that you was willing to drop the non-profit use restriction
but it seems I was wrong. The reason that you did not found a standard
Open Source license is that your restriction is in conflict per
definition with Open Source licenses.
I had this discussion with Joe Felsenstein for years and he finally
realised that the non-commercial restriction has more drawbacks (like
beeing not included in any Linux distribution) than advantages (really
low payment from commercial users ... are you sure that commercial
users really pay for a license in their closed products?)
This idea has distributed quite widely in scientific software world
and may be you reconsider your decision.
Kind regards
Andreas.

Post by Jim Kent
Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

part

less

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

are

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Maximilian Haeussler

2017-10-25 20:23:42 UTC

Is there any chance to get it into the non-free repo?

Debian has other non-OSS software packages that it distributes as part
of non-free. Why not BLAT?

Post by Andreas Tille
Hi Kent,
thanks for your quick reply.

I think I explained in the past in detail that *any* restriction for
*any* user makes the code non-free and can not distributed with Debian
(and in the same manner not with seqtools which is GPL-3). I had the
impression that you was willing to drop the non-profit use restriction
but it seems I was wrong. The reason that you did not found a standard
Open Source license is that your restriction is in conflict per
definition with Open Source licenses.
I had this discussion with Joe Felsenstein for years and he finally
realised that the non-commercial restriction has more drawbacks (like
beeing not included in any Linux distribution) than advantages (really
low payment from commercial users ... are you sure that commercial
users really pay for a license in their closed products?)
This idea has distributed quite widely in scientific software world
and may be you reconsider your decision.
Kind regards
Andreas.

Post by Jim Kent
Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

part

less

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

are

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Jim Kent

2017-10-25 20:59:23 UTC

We attempted to get it into in the non-free section last time, and
that is where we still ran into problems where the Debian folks didn't
like our usual license, I went through a web site to find a standard
license for noncommercial use that they could agree with, and then
they couldn't work with what I'd found there for a reason that seemed
like a lawyer's or ontologist's nits, and it seemed like it would be a
time consuming and potentially expensive thing to find a solution that
would make everyone happy.

At any rate, this current thread seems to be based on us no longer
charging for or restricting commercial use, which is false. Our
license is still same as ever.

Post by Maximilian Haeussler
Is there any chance to get it into the non-free repo?
Debian has other non-OSS software packages that it distributes as part
of non-free. Why not BLAT?

Post by Andreas Tille
Hi Kent,
thanks for your quick reply.

I think I explained in the past in detail that *any* restriction for
*any* user makes the code non-free and can not distributed with Debian
(and in the same manner not with seqtools which is GPL-3). I had the
impression that you was willing to drop the non-profit use restriction
but it seems I was wrong. The reason that you did not found a standard
Open Source license is that your restriction is in conflict per
definition with Open Source licenses.
I had this discussion with Joe Felsenstein for years and he finally
realised that the non-commercial restriction has more drawbacks (like
beeing not included in any Linux distribution) than advantages (really
low payment from commercial users ... are you sure that commercial
users really pay for a license in their closed products?)
This idea has distributed quite widely in scientific software world
and may be you reconsider your decision.
Kind regards
Andreas.

Post by Jim Kent
Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

part

less

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

are

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Maximilian Haeussler

2017-10-25 21:03:23 UTC

Andreas: If clustal, cufflinks, phylip, paml and many other
bioinformatics packages are just fine in the non-free repo, couldn't
we do the same with blat?

http://http.us.debian.org/debian/pool/non-free/c/
http://http.us.debian.org/debian/pool/non-free/e/
http://http.us.debian.org/debian/pool/non-free/p/

Post by Jim Kent
We attempted to get it into in the non-free section last time, and
that is where we still ran into problems where the Debian folks didn't
like our usual license, I went through a web site to find a standard
license for noncommercial use that they could agree with, and then
they couldn't work with what I'd found there for a reason that seemed
like a lawyer's or ontologist's nits, and it seemed like it would be a
time consuming and potentially expensive thing to find a solution that
would make everyone happy.
At any rate, this current thread seems to be based on us no longer
charging for or restricting commercial use, which is false. Our
license is still same as ever.

Post by Maximilian Haeussler
Is there any chance to get it into the non-free repo?
Debian has other non-OSS software packages that it distributes as part
of non-free. Why not BLAT?

Post by Andreas Tille
Hi Kent,
thanks for your quick reply.

I think I explained in the past in detail that *any* restriction for
*any* user makes the code non-free and can not distributed with Debian
(and in the same manner not with seqtools which is GPL-3). I had the
impression that you was willing to drop the non-profit use restriction
but it seems I was wrong. The reason that you did not found a standard
Open Source license is that your restriction is in conflict per
definition with Open Source licenses.
I had this discussion with Joe Felsenstein for years and he finally
realised that the non-commercial restriction has more drawbacks (like
beeing not included in any Linux distribution) than advantages (really
low payment from commercial users ... are you sure that commercial
users really pay for a license in their closed products?)
This idea has distributed quite widely in scientific software world
and may be you reconsider your decision.
Kind regards
Andreas.

Post by Jim Kent
Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

part

less

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

are

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Afif Elghraoui

2017-10-25 21:50:16 UTC

Hello,

Post by Maximilian Haeussler
Andreas: If clustal, cufflinks, phylip, paml and many other
bioinformatics packages are just fine in the non-free repo, couldn't
we do the same with blat?
http://http.us.debian.org/debian/pool/non-free/c/
http://http.us.debian.org/debian/pool/non-free/e/
http://http.us.debian.org/debian/pool/non-free/p/

Just to clarify Andreas' statement about it not being allowed in Debian. non-free is not technically part of Debian: packages are not autobuilt there, no reproducibility checking or continuous integration is done on them, and even any free-software package that depends on it is similarly excluded from Debian main.

About the lack of autobuilding--this means only whatever binaries the package maintainer uploads will be available, and any rebuilds that need to happen must be done manually by the maintainer.

For non-free, the software just needs to be legally redistributable

regards
Afif

Steffen Möller

2017-10-26 00:20:11 UTC

Hello,

Post by Afif Elghraoui
Hello,

Post by Maximilian Haeussler
Andreas: If clustal,

When I had asked Andreas in 2001 or so to update clustalw for me, this
was what started my contributions to Debian. Maybe this qualifies me to
join this somewhat unfortunate thread.

Post by Afif Elghraoui

Post by Maximilian Haeussler
cufflinks, phylip, paml and many other
bioinformatics packages are just fine in the non-free repo, couldn't
we do the same with blat?
http://http.us.debian.org/debian/pool/non-free/c/
http://http.us.debian.org/debian/pool/non-free/e/
http://http.us.debian.org/debian/pool/non-free/p/

Just to clarify Andreas' statement about it not being allowed in Debian. non-free is not technically part of Debian: packages are not autobuilt there,

Actually, once upon a time they were autobuilt from a subset of Debian's
auto-builders.

Post by Afif Elghraoui
no reproducibility checking or continuous integration is done on them, and even any free-software package that depends on it is similarly excluded from Debian main.
About the lack of autobuilding--this means only whatever binaries the package maintainer uploads will be available, and any rebuilds that need to happen must be done manually by the maintainer.
For non-free, the software just needs to be legally redistributable

Yes, and I admit to have thought that Debian would ship blat already
(non-free or not). A quick google search found
https://lists.debian.org/debian-med/2014/02/msg00264.html from which I
understand that you have packages prepared.

Your first upload needs to be done by a Debian Developer. This could for
instance be Andreas, Afif or me or some 1000 other people. Your sponsor
checks that the package is OK to be redistributed, that the package
builds, that there is a description of the package and then upload. For
updates of blat it would be nice if you would find a volunteer among
your team to perform those updates independently from any sponsor. For
that "Debian Maintainer" is the hurdle is that you have two Debian
Developers signing a GPG key of yours, which means that you have met
them personally. There is a bunch of them on every ISMB and other
conferences, so this should not be too difficult or you find someone on
https://wiki.debian.org/Keysigning/Offers#US .

Please post a pointer to your package on this list or email me directly
and we get this going.

Steffen

Andreas Tille

2017-10-26 06:49:55 UTC

Post by Maximilian Haeussler

Hi Maximilian,

Post by Maximilian Haeussler
Andreas: If clustal,

Wrong, the latest clustal is all in main - you are refering to some old
clustalw-mpi which is only in oldstable and was removed from current
distributions. Clustal is free since a long time.

Post by Maximilian Haeussler
cufflinks,

That's due to the use of locfit which has in principle a free license
(specifically in terms of beeing free to distribute) but a restriction
that it ist restricting benchmarks. (BTW, I'm trying since years to
reach the author who has put this strange restriction but failed. :-()

Post by Maximilian Haeussler
phylip,

Is in main since some years. I gave the example that I discussed with
the author for years.

Post by Maximilian Haeussler
paml

I also had a long dispute with the author and it is now free.

Post by Maximilian Haeussler
and many other
bioinformatics packages are just fine in the non-free repo, couldn't
we do the same with blat?
http://http.us.debian.org/debian/pool/non-free/c/
http://http.us.debian.org/debian/pool/non-free/e/
http://http.us.debian.org/debian/pool/non-free/p/

While your examples are mostly not correct if you want a real list of
non-free software in Debian have a look here:

https://blends.debian.org/med/tasks/bio#non-free-debs

But the software there has very different reasons for beeing in non-free
or contrib. For instance in the case of igv it is just since nobody was
able to package all binary JARs into separate packages containing the
source and enables to build the binary from it. Similarly with ugene
which simply contains a lot of third party software that needs to be
ironed out and a license review has to be done. Sometimes it just costs
a lot of time to make some software acceptable for Debian main.

If you want to know what we are doing to convince authors to free their
code here is a list

https://wiki.debian.org/DebianMed/SoftwareLiberation

Finally, if you wonder whether blat could be in non-free - provide that
the distribution is permitted which is currently not (see my other mail)
- yes for sure it could if somebody would do the packaging work. As it
concerns me I have *lots* of Free Software on my desk and I give this
preference since I consider these the technically (from a distribution
point of view!) more valuable projects since I have proper quality
assurance means. For non-free software it is rather providing an easy
way of installation for those who have no clue to use the installation
method the authors are providing. I consider this advantage as to
small compared to what I can do otherwise in my spare time. But if
somebody (like you?) would take up my previous packaging attempt and
create a blat package for non-free and if the license permits I would
surely check that work and sponsor it to the Debian mirror. Everybody
is free to work on Debian packages and the Debian Med team has quite
some record in teaching newcomers how it works[1].

Kind regards

Andreas.

[1] https://wiki.debian.org/DebianMed/MoM

Post by Maximilian Haeussler
Is there any chance to get it into the non-free repo?
Debian has other non-OSS software packages that it distributes as part
of non-free. Why not BLAT?

Post by Andreas Tille
Hi Kent,
thanks for your quick reply.

I think I explained in the past in detail that *any* restriction for
*any* user makes the code non-free and can not distributed with Debian
(and in the same manner not with seqtools which is GPL-3). I had the
impression that you was willing to drop the non-profit use restriction
but it seems I was wrong. The reason that you did not found a standard
Open Source license is that your restriction is in conflict per
definition with Open Source licenses.
I had this discussion with Joe Felsenstein for years and he finally
realised that the non-commercial restriction has more drawbacks (like
beeing not included in any Linux distribution) than advantages (really
low payment from commercial users ... are you sure that commercial
users really pay for a license in their closed products?)
This idea has distributed quite widely in scientific software world
and may be you reconsider your decision.
Kind regards
Andreas.

Post by Jim Kent
Yes. If you could reiterate some of the links, or if you prefer just
forward this whole thread to Hiram, it would be great. Then I can go back
to wrestling with the ENCODE monster!

Disabling the pslCheck seems like the sensible and pragmatic thing.
I'm going to write a short note introducing you to Hiram Clawson, and

also

Ann Zweig our project manager (and Hiram's boss). It is actually part of
our grant to package the tools in ways to make it easier for people to

use

them. Our current system is not so bad, but it requires people to
actually read the README, and set an environment variable. This was

state

part

less

Post by Andreas Tille
Hi Jim,

I'm glad you isolated it to the -O2.

:-)

I don't think there's a super easy way to cut pslCheck out of the

whole

1,200,000 UCSC genomics source tree.

For the moment I simply disabled this check. I guess it is also this
way sufficient to detect potential problems (and it was not the

pslCheck

Post by Andreas Tille
that failed in the first place).

I would, on the other hand, be very
happy for you to take on the job of packaging up that whole source

tree

Debian. I could refer you to a less busy member of my staff, Hiram
Clawson, who has a _lot_ of experience helping people get that to

build.

are

The licensing of it is quite complex alas. There are three main
- a small part which is owned by me in a directory called jkOwnLib,

and

the blat directories

This would probably make a separate library package. However, you

might

Post by Andreas Tille
consider a name which is more descriptive than jkOwnLib.

- a medium sized part that contains stuff we regard as generally

useful

which is essentially public domain, but that we are happy distributed

under

a BSD or MIT license

Cool. That would be very interesting.

- a large part that is genomics in general, and UCSC Genome Browser

particular specific that is owned by UCSC and has a license much like

blat

- free for personal, academic, and non-profit use, and requiring a

license

for commercial use. In this case the licence needs to come from UCSC
(contact Will Hale) rather than Kent Informatics (contact Heidi

Brumbaugh).
In case we have a good plan about the technical details we should
probable contact these persons regarding the licensing.
Kind regards
Andreas.
--
http://fam-tille.de

--
http://fam-tille.de

Andreas Tille

2017-10-26 06:16:35 UTC

Post by Maximilian Haeussler
Is there any chance to get it into the non-free repo?

https://lists.alioth.debian.org/pipermail/debian-med-packaging/2014-March/025525.html

Post by Maximilian Haeussler
Debian has other non-OSS software packages that it distributes as part
of non-free. Why not BLAT?

Strictly speaking non-free is not a part of Debian and there are lots of
drawbacks for packages in non-free which might not be visible for a lot
of people. Its not only that its not autobuilt on all architectures it
also does not get sufficient qualilty assurance support like automatic
testing in Contiuous Integration. For me this make it not a good target
for scientific software.

Kind regards

Andreas.

--
http://fam-tille.de

Jim Kent

2014-02-27 08:17:09 UTC