5 years agomain release 1.4 Release_1.4
Michael Peter Christen [Fri, 15 Mar 2013 09:25:47 +0000 (10:25 +0100)]
main release 1.4

5 years agofixed NPE during index abstract computation
Michael Peter Christen [Fri, 15 Mar 2013 09:04:27 +0000 (10:04 +0100)]
fixed NPE during index abstract computation

5 years agoadded a restart hint
Michael Peter Christen [Fri, 15 Mar 2013 09:00:06 +0000 (10:00 +0100)]
added a restart hint

5 years agoturned severe message to warning message about network failure events
Michael Peter Christen [Fri, 15 Mar 2013 08:40:02 +0000 (09:40 +0100)]
turned severe message to warning message about network failure events

5 years agofix for wrong class name in log
Michael Peter Christen [Fri, 15 Mar 2013 08:35:57 +0000 (09:35 +0100)]
fix for wrong class name in log

5 years ago- do not create a new query for all remote peers
Michael Peter Christen [Thu, 14 Mar 2013 23:14:28 +0000 (00:14 +0100)]
- do not create a new query for all remote peers
- no document search this time
- adjusted banner and network to not show 'WORDS' but DHT Chunks. This
is to avoid confusion for robinson peers which do not create Word

5 years agouse appropriate ranking for each search situation:
Michael Peter Christen [Thu, 14 Mar 2013 20:13:12 +0000 (21:13 +0100)]
use appropriate ranking for each search situation:
- when using the /date modifier, a date ranking profile is used
- when using a site: modifier, a ranking profile supporting longer urls
is used

5 years agoadded all clickdepth computations for source and target paths in
Michael Peter Christen [Thu, 14 Mar 2013 16:54:33 +0000 (17:54 +0100)]
added all clickdepth computations for source and target paths in
webstructure core

5 years agorefactoring of clickdepth computation as preparation for clickdepth
Michael Peter Christen [Thu, 14 Mar 2013 11:13:02 +0000 (12:13 +0100)]
refactoring of clickdepth computation as preparation for clickdepth
computation of webgraph links

5 years agoremoved unused tag fields
Michael Peter Christen [Thu, 14 Mar 2013 09:35:21 +0000 (10:35 +0100)]
removed unused tag fields

5 years agofix error msg in ConfigHeuristics_p
reger [Thu, 14 Mar 2013 02:30:25 +0000 (03:30 +0100)]
fix error msg in ConfigHeuristics_p

5 years agoadjust Opensearch discover function to new webgraph Solr schema
reger [Thu, 14 Mar 2013 02:10:54 +0000 (03:10 +0100)]
adjust Opensearch discover function to new webgraph Solr schema

5 years agoadded clickdepth field writing for webgraph core (unfinished)
orbiter [Thu, 14 Mar 2013 00:35:38 +0000 (01:35 +0100)]
added clickdepth field writing for webgraph core (unfinished)

5 years agofix for possible memory leaks
orbiter [Wed, 13 Mar 2013 16:55:37 +0000 (17:55 +0100)]
fix for possible memory leaks

5 years agochanges in ranking computation
Michael Peter Christen [Wed, 13 Mar 2013 13:47:00 +0000 (14:47 +0100)]
changes in ranking computation
- an existing ranking servlet for solr was extended. It is now possible
to set boost values for fields, boost functions and boost queries.
- The ranking can have different instances, but currently only the first
one is used
- added an abstraction layer for fields which can be used for search and
those fields can be edited in the solr ranking configruation
- the ranking value from solr within the field score is used to combine
remote search requests, which all are created using the same locally
defined boost values
- reduced the number of fields which are used for search (makes it
- replaced some text fields by string fields (makes indexing faster)
- removed classes which had no use
- made a large number of experiments for a better ranking and created a
temporary setting which prefers hits inside titles
- adjusted also the RWI-based ranking computation to 'prefer title'
- made special cases like for portal search where no post-processing and
post-ranking is wanted: this keeps the original ranking order as done by
- fixed many bugs with old settings for ranking

5 years agoset RootNodeFlag only if EmbeddedSolr is connected (as RootNodes may receive direct...
reger [Tue, 12 Mar 2013 02:13:14 +0000 (03:13 +0100)]
set RootNodeFlag only if EmbeddedSolr is connected (as RootNodes may receive direct Solr queries)

5 years agoMerge branch 'master' of git://gitorious.org/yacy/rc1.git
reger [Tue, 12 Mar 2013 01:51:17 +0000 (02:51 +0100)]
Merge branch 'master' of git://gitorious.org/yacy/rc1.git

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
orbiter [Mon, 11 Mar 2013 17:23:43 +0000 (18:23 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years agoremoved target_tag_s (superfluous)
Michael Peter Christen [Mon, 11 Mar 2013 09:46:29 +0000 (10:46 +0100)]
removed target_tag_s (superfluous)

5 years agofix for NPE if surrogates do not exist
orbiter [Sun, 10 Mar 2013 18:46:06 +0000 (19:46 +0100)]
fix for NPE if surrogates do not exist

5 years agoreplace the terminateOldSessions - return immediate time from fixed 3 sec to request...
reger [Sun, 10 Mar 2013 04:22:18 +0000 (05:22 +0100)]
replace the terminateOldSessions - return immediate time from fixed 3 sec to  requested minage parameter

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Michael Peter Christen [Sun, 10 Mar 2013 01:29:38 +0000 (02:29 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years ago- added more selection criteria for network seed list
Michael Peter Christen [Sun, 10 Mar 2013 01:26:24 +0000 (02:26 +0100)]
- added more selection criteria for network seed list
- enhanced up script

5 years agoadded/fixed missing DOCTYPE line (submitted by Thomas)
orbiter [Fri, 8 Mar 2013 13:40:09 +0000 (14:40 +0100)]
added/fixed missing DOCTYPE line (submitted by Thomas)

5 years agofix for wrong mime type in noload crawler
Michael Peter Christen [Thu, 7 Mar 2013 14:31:00 +0000 (15:31 +0100)]
fix for wrong mime type in noload crawler

5 years agofixes to search debugging after testing with the different search
Michael Peter Christen [Tue, 5 Mar 2013 20:28:22 +0000 (21:28 +0100)]
fixes to search debugging after testing with the different search
debugging options

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Michael Peter Christen [Tue, 5 Mar 2013 11:24:10 +0000 (12:24 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years agoconcurrent snippet fetching from solr results which do not have snippets
Michael Peter Christen [Tue, 5 Mar 2013 11:24:01 +0000 (12:24 +0100)]
concurrent snippet fetching from solr results which do not have snippets

5 years agoadded debug switches for detailed search testing
orbiter [Tue, 5 Mar 2013 11:19:32 +0000 (12:19 +0100)]
added debug switches for detailed search testing

5 years agoadded filter queries for better image, audio and video results
orbiter [Mon, 4 Mar 2013 20:18:54 +0000 (21:18 +0100)]
added filter queries for better image, audio and video results

5 years agoadded missing cleanup statements for short memory cases during search
Michael Peter Christen [Mon, 4 Mar 2013 12:01:24 +0000 (13:01 +0100)]
added missing cleanup statements for short memory cases during search

5 years agodo not put the fulltext field text_t into the search cache because it is
orbiter [Mon, 4 Mar 2013 11:01:10 +0000 (12:01 +0100)]
do not put the fulltext field text_t into the search cache because it is
not used there and uses a lot of memory

5 years agoin method exists() also use the new caching-stacks for
Michael Peter Christen [Mon, 4 Mar 2013 00:13:17 +0000 (01:13 +0100)]
in method exists() also use the new caching-stacks for

5 years agoenhanced the search result processing
Michael Peter Christen [Sun, 3 Mar 2013 23:17:29 +0000 (00:17 +0100)]
enhanced the search result processing
- no waiting time at the end
- switched on 'classic' snippet production and verification (again)

5 years agoDHT-transferred metadata and crawl receipts now also use the delayed
Michael Peter Christen [Sun, 3 Mar 2013 23:07:52 +0000 (00:07 +0100)]
DHT-transferred metadata and crawl receipts now also use the delayed
search cache to prevent that too much IO load is on the peer during

5 years agobetter protection against OOM during search flush and fixed missing
Michael Peter Christen [Sun, 3 Mar 2013 22:45:47 +0000 (23:45 +0100)]
better protection against OOM during search flush and fixed missing
result push

5 years ago- enhanced concurrency during search without IO blocking
Michael Peter Christen [Sun, 3 Mar 2013 21:38:50 +0000 (22:38 +0100)]
- enhanced concurrency during search without IO blocking
- introduced a second queue to flush remote search results (now: old
metadata structure from DHT peers)
- fixed result counters

5 years ago*) For some reason this seems to fix a ClassCastException on my system
Marc Nause [Sun, 3 Mar 2013 19:38:20 +0000 (20:38 +0100)]
*) For some reason this seems to fix a ClassCastException on my system

5 years agomade index storage from DHT search result concurrently. This prevents
Michael Peter Christen [Sat, 2 Mar 2013 09:25:52 +0000 (10:25 +0100)]
made index storage from DHT search result concurrently. This prevents
blocking by high CPU usage during search. Also: removed query from Solr
for DHT search results; results are taken from the pending queue.

5 years agofix for search
orbiter [Fri, 1 Mar 2013 18:18:16 +0000 (19:18 +0100)]
fix for search

5 years ago- enhanced solr.add procedure for mass adds
orbiter [Fri, 1 Mar 2013 14:27:17 +0000 (15:27 +0100)]
- enhanced solr.add procedure for mass adds
- removed unused solr access classes
- made snippet generation for documents aus YaCy RWI/DHT concurrent (as
it was before the search process removation)
- reduced the number of remote results in settings file because the
processing of such mass documents add is too CPU-intensive (in Solr)

5 years agofix of page navigation for formatted totalcount numbers
orbiter [Thu, 28 Feb 2013 23:48:28 +0000 (00:48 +0100)]
fix of page navigation for formatted totalcount numbers

5 years agoÜbersetzung des Domain Navigators als Anbieter Navigator; ist als Nutzen
orbiter [Thu, 28 Feb 2013 22:55:46 +0000 (23:55 +0100)]
Übersetzung des Domain Navigators als Anbieter Navigator; ist als Nutzen
besser erklärbar

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Michael Peter Christen [Thu, 28 Feb 2013 14:55:13 +0000 (15:55 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years agobetter/less requests to local solr; the request is made in chunks which
orbiter [Thu, 28 Feb 2013 13:04:08 +0000 (14:04 +0100)]
better/less requests to local solr; the request is made in chunks which
are exactly at only that size which is needed to present the current
search result page. This will also cause that next solr request are made
automatically during switching to next pages.

5 years agodisabled clickdepth computation during craling since that is repeated
Michael Peter Christen [Thu, 28 Feb 2013 01:25:39 +0000 (02:25 +0100)]
disabled clickdepth computation during craling since that is repeated
during clean-up phase.

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
orbiter [Wed, 27 Feb 2013 21:44:57 +0000 (22:44 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years agocorrected result counter
orbiter [Wed, 27 Feb 2013 21:40:23 +0000 (22:40 +0100)]
corrected result counter

5 years agoremoved the dns prefetch because that was not soo useful
orbiter [Wed, 27 Feb 2013 19:58:34 +0000 (20:58 +0100)]
removed the dns prefetch because that was not soo useful

5 years agoadded recrawl/reload to CrawlStartSite for a timeout of 3 days
orbiter [Wed, 27 Feb 2013 10:43:36 +0000 (11:43 +0100)]
added recrawl/reload to CrawlStartSite for a timeout of 3 days

5 years agoadded option to create empty vocabularies
orbiter [Wed, 27 Feb 2013 07:24:37 +0000 (08:24 +0100)]
added option to create empty vocabularies

5 years agofix link to IndexSchema_p.html
bubu [Tue, 26 Feb 2013 20:12:44 +0000 (21:12 +0100)]
fix link to IndexSchema_p.html

5 years agoremoved size request
Michael Peter Christen [Tue, 26 Feb 2013 16:53:44 +0000 (17:53 +0100)]
removed size request

5 years agocomplete redesign of search process:
Michael Peter Christen [Tue, 26 Feb 2013 16:16:31 +0000 (17:16 +0100)]
complete redesign of search process:
- removed 'worker' processes
- no internal time-out behaviour: methods either are successful or
return null
- waiting is only done on top-level
- removed snippet-production; this is replaced by solr snippets
- removed statistics based on solr size queries (they had been VERY
long); the statistics (like suggestions or tag cloud) are now again
based on the old but very fast RWI index. In portal or intranet mode the
RWI index is usually switched off; if you like to have statistics again
then you must switch on the rwis again in this mode.
- fixed many bugs regarding correct page counter

5 years agotesting to use solr for portalsearch caused some bugfixing but no full
Michael Peter Christen [Mon, 25 Feb 2013 13:31:50 +0000 (14:31 +0100)]
testing to use solr for portalsearch caused some bugfixing but no full
success: try to comment out the solr search request in

5 years agofix for schema export to consider also automatically generated
Michael Peter Christen [Mon, 25 Feb 2013 00:13:03 +0000 (01:13 +0100)]
fix for schema export to consider also automatically generated
coordinate fields

5 years ago- generalized SchemaConfiguration into super-class Configuration and
Michael Peter Christen [Sun, 24 Feb 2013 23:09:41 +0000 (00:09 +0100)]
- generalized SchemaConfiguration into super-class Configuration and
adopted other classes which used the configuration-only access for that
- removed many warnings
- adjusted logging

5 years agofix for webgraph delete query
Michael Peter Christen [Sun, 24 Feb 2013 17:17:58 +0000 (18:17 +0100)]
fix for webgraph delete query

5 years ago- added flags in IndexFederated_p.html to switch on or off the webgraph
Michael Peter Christen [Sun, 24 Feb 2013 17:09:34 +0000 (18:09 +0100)]
- added flags in IndexFederated_p.html to switch on or off the webgraph
index (new solr core webgraph) .. this is now off by default
- completely redesigned this servlet
- added description how to attach a remote solr
- adjusted naming of servlet and menues
- moved 'lazy initialization' attribut from IndexSchema to
IndexFederated (this is a general option) back again.

5 years ago- Removed log4j from libraries. This can be removed because the package
Michael Peter Christen [Sat, 23 Feb 2013 15:45:05 +0000 (16:45 +0100)]
- Removed log4j from libraries. This can be removed because the package
log4j-over-slf4j is there. From slf4j all loggings are routed to the jdk
logger. Now all loggings are consistently done to the jdk logger.
- added some lines to the logging properties to suppress many solr
logging statements. The number of the logging entries had already become
a performance issue, therefore removing these from the log should
increase performance.

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Michael Peter Christen [Sat, 23 Feb 2013 14:55:21 +0000 (15:55 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
orbiter [Sat, 23 Feb 2013 13:33:29 +0000 (14:33 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years agoupdated wstx-asl to 3.2.9
orbiter [Sat, 23 Feb 2013 13:33:17 +0000 (14:33 +0100)]
updated wstx-asl to 3.2.9

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Michael Peter Christen [Sat, 23 Feb 2013 07:48:33 +0000 (08:48 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git


also moved portalsearch-dev to yacy-portalsearch to be able to fix
problems with new attachment to solr of the search widget

5 years agofixes to schema
Michael Peter Christen [Sat, 23 Feb 2013 07:14:10 +0000 (08:14 +0100)]
fixes to schema

5 years agoon remote Solr search take only locally enabled schema fields from remote solrdocumen...
reger [Fri, 22 Feb 2013 21:17:45 +0000 (22:17 +0100)]
on remote Solr search take only locally enabled schema fields from remote solrdocument for the inputdocument added to local index

5 years agoremove obsolete Solr "commit within" input field from IndexFederated
reger [Fri, 22 Feb 2013 21:03:32 +0000 (22:03 +0100)]
remove obsolete Solr "commit within" input field from IndexFederated
see https://gitorious.org/yacy/rc1/commit/41116066548be3d7987d7eaa73f2aac43e6f1e43

5 years agoadded the generation of 50 (!!) new solr field in the core 'webgraph'.
Michael Peter Christen [Fri, 22 Feb 2013 14:45:15 +0000 (15:45 +0100)]
added the generation of 50 (!!) new solr field in the core 'webgraph'.
The default schema uses only some of them and the resting search index
has now the following properties:
- webgraph size will have about 40 times as much entries as default
- the complete index size will increase and may be about the double size
of current amount
As testing showed, not much indexing performance is lost. The default
index will be smaller (moved fields out of it); thus searching
can be faster.
The new index will cause that some old parts in YaCy can be removed,
i.e. specialized webgraph data and the noload crawler. The new index
will make it possible to:
- search within link texts of linked but not indexed documents (about 20
times of document index in size!!)
- get a very detailed link graph
- enhance ranking using a complete link graph

To get the full access to the new index, the API to solr has now two
access points: one with attribute core=collection1 for the default
search index and core=webgraph to the new webgraph search index. This is
also avaiable for p2p operation but client access is not yet

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Michael Peter Christen [Thu, 21 Feb 2013 12:24:10 +0000 (13:24 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years agointroduced a second core named 'webgraph'. This core will hold the link
Michael Peter Christen [Thu, 21 Feb 2013 12:23:55 +0000 (13:23 +0100)]
introduced a second core named 'webgraph'. This core will hold the link
structure, but is not filled yet. To have the opportunity of a second
core, multi-core functionality had to be implemented to the
deep-embedded solr:
- migrated the solr_40 directory content to a subdirectory
'collection1'; the previously used default core is now called
- added solr_40/webgraph subdirectory as second core
- added a servlet configuration for the second core 'webgraph' in
- added instance handling as addition to solr connections: all solr
connectors are now instances of an solr 'instance' object; this required
a complete re-design of the solr embedding
- migrated also caching and sharding ontop of new instance handling
- migrated the search apis to handle now the access to a specific core,
the default core named 'collection1'
- migrated the remote solr search interface to access shards of cores;
for the yacy remote search the default core is now called 'solr'; using
the peer address as solr address
- migrated the solr backup and restore process: old backups cannot be
used after this migration!
- redesign of solr instance handling in all methods which access the
instances: they cannot hold copies of these instances any more; the must
retrieve the actuall connection object every time they want to write to
it (this solves also some bugs when switching the index/network)
- added another schema 'solr.webgraph.schema', the old solr.keys.list is
replaced by solr.collection.schema

5 years agoremove CPGEN from Windows batch files
reger [Sun, 17 Feb 2013 02:26:46 +0000 (03:26 +0100)]
remove CPGEN from Windows batch files
(classpath for all needed libraries is defined in manifest  of yacycore.jar)

5 years agofixed interactive search which caused an error if pubDate is not present
orbiter [Sat, 16 Feb 2013 19:33:27 +0000 (20:33 +0100)]
fixed interactive search which caused an error if pubDate is not present
in a search result

5 years agoprevent that crawl starts with very large url lists cause a time-out in
Michael Peter Christen [Fri, 15 Feb 2013 00:58:28 +0000 (01:58 +0100)]
prevent that crawl starts with very large url lists cause a time-out in
the user front-end

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Michael Peter Christen [Fri, 15 Feb 2013 00:38:23 +0000 (01:38 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years agoFull redesign of solr connection architecture. This was done to support
Michael Peter Christen [Fri, 15 Feb 2013 00:38:10 +0000 (01:38 +0100)]
Full redesign of solr connection architecture. This was done to support
multiple solr cores instead of just one. Therefore it is now necessary
to distuingish between solr server connections (called an 'Instance')
and a connection to a single solr core. One Instance may now have
multiple connector classes assigned to it, each connecting to a single
To support multiple cores it is also necessary to distinguish between
the connection configuration and the configuration of the index schema.
We will have multiple schema configurations in the future, each for
every solr core. This caused that the IndexFederated servlet had to be
split into two parts, the new Servlet for the Schema editor is now in
the IndexSchema Servlet.

5 years agoMerge branch 'master' of git@gitorious.org:yacy/rc1.git
Marc Nause [Wed, 13 Feb 2013 18:31:12 +0000 (19:31 +0100)]
Merge branch 'master' of git@gitorious.org:yacy/rc1.git

5 years ago*) removed Skype online indicator (was not working anymore)
Marc Nause [Wed, 13 Feb 2013 18:29:40 +0000 (19:29 +0100)]
*) removed Skype online indicator (was not working anymore)
*) updated ICQ URLs

5 years agoremoved the commitWithin attribute because that is not the way how the
Michael Peter Christen [Wed, 13 Feb 2013 01:29:47 +0000 (02:29 +0100)]
removed the commitWithin attribute because that is not the way how the
index is updated the right way for us. May also be be superfluous with
the solr 4.0 softcommit.

5 years agofix to unbalanced tag and license for null objects
Michael Peter Christen [Wed, 13 Feb 2013 00:23:05 +0000 (01:23 +0100)]
fix to unbalanced tag and license for null objects

5 years agoadded jsonp option to yjson result writer
Michael Peter Christen [Wed, 13 Feb 2013 00:11:57 +0000 (01:11 +0100)]
added jsonp option to yjson result writer

5 years agoAdded image license generation for solr image search results when
Michael Peter Christen [Tue, 12 Feb 2013 23:33:53 +0000 (00:33 +0100)]
Added image license generation for solr image search results when
results are generated within yjson result writer. This makes it possible
to view images in yacyinteractive from solr.

5 years agofixed json search, quotes, auto-facets, urls etc. for
Michael Peter Christen [Tue, 12 Feb 2013 23:01:38 +0000 (00:01 +0100)]
fixed json search, quotes, auto-facets, urls etc. for

5 years agoMoved methods from SolrServerConnector to AbstractSolrConnector with the
Michael Peter Christen [Tue, 12 Feb 2013 21:03:10 +0000 (22:03 +0100)]
Moved methods from SolrServerConnector to AbstractSolrConnector with the
result that most of these methods become superfluous in other classes.
This is a generalization step towards multi-indexes in Solr.

5 years agobetter filesearch layout
Michael Peter Christen [Tue, 12 Feb 2013 11:21:29 +0000 (12:21 +0100)]
better filesearch layout

5 years agoreduced number of facets in yacyinteractive (only filetype necessary)
Michael Peter Christen [Tue, 12 Feb 2013 11:00:54 +0000 (12:00 +0100)]
reduced number of facets in yacyinteractive (only filetype necessary)

5 years agoreverted put-semantics back to as-usual in serverObjects and introduced
Michael Peter Christen [Tue, 12 Feb 2013 10:52:33 +0000 (11:52 +0100)]
reverted put-semantics back to as-usual in serverObjects and introduced
an add-method to put in several objects for the same key

5 years agoMerge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Michael Peter Christen [Tue, 12 Feb 2013 02:42:58 +0000 (03:42 +0100)]
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git

5 years agoextended JSON Response Writer and Opensearch Response Writer for the
Michael Peter Christen [Tue, 12 Feb 2013 02:42:46 +0000 (03:42 +0100)]
extended JSON Response Writer and Opensearch Response Writer for the
Solr search interface in such way that it is possible to use this
interface for the yacyinteractive search. This search interface is now
much faster using the Solr search directly. For the Solr interface it
was necessary to create a translation from the YaCy search modifiers to
the Solr facet selection. This was added in such a way that it becomes
generic for the normal YaCy search and as a on-top evaluation for Solr

5 years agomake sure yacy.running is deleted if not running (catch exception)
reger [Mon, 11 Feb 2013 21:53:19 +0000 (22:53 +0100)]
make sure yacy.running is deleted if not running (catch exception)
- to prevent following log if YaCy was previously not properly shutdown

E ... STARTUP WARNING: the file C:\src\git\yacy-rc1\DATA\yacy.running exists, this usually means that a YaCy instance is still running
E ... STARTUP FATAL ERROR: java.util.concurrent.TimeoutException
java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException
at net.yacy.cora.protocol.TimeoutRequest.call(TimeoutRequest.java:91)
at net.yacy.cora.protocol.TimeoutRequest.ping(TimeoutRequest.java:112)
at net.yacy.yacy.startup(yacy.java:200)
at net.yacy.yacy.main(yacy.java:638)
Caused by: java.util.concurrent.TimeoutException

- adjust Netbeans path (to solr4.1.jars)

5 years agoextended the serverObjects to be able to hold multipel values for a
Michael Peter Christen [Mon, 11 Feb 2013 21:12:15 +0000 (22:12 +0100)]
extended the serverObjects to be able to hold multipel values for a
single key. This is done using the solr class MultiMapSolrParams. That
class is needed in the OpensearchResultWriter to get multiple facet

5 years agoadded more metadata fields and facets to OpensearchResponseWriter.
Michael Peter Christen [Mon, 11 Feb 2013 21:10:14 +0000 (22:10 +0100)]
added more metadata fields and facets to OpensearchResponseWriter.
This should make it possible to replace the original and enriched yacy
opensearch result with a solr output in opensearch format.

5 years agofix in html parser and bookmark generation
Michael Peter Christen [Mon, 11 Feb 2013 12:28:08 +0000 (13:28 +0100)]
fix in html parser and bookmark generation

5 years agomoved bookmarks back to more prominent location (even if this does not
Michael Peter Christen [Sat, 9 Feb 2013 05:57:20 +0000 (06:57 +0100)]
moved bookmarks back to more prominent location (even if this does not
fit to the 'Search Interfaces' headline)

5 years agobetter error handling for bookmarks
Michael Peter Christen [Sat, 9 Feb 2013 05:55:57 +0000 (06:55 +0100)]
better error handling for bookmarks

5 years agowhen searching the network, do not search on robinson peers with the old
Michael Peter Christen [Fri, 8 Feb 2013 17:30:08 +0000 (18:30 +0100)]
when searching the network, do not search on robinson peers with the old
DHT search interface. Now use the solr interface.

5 years agoA robinson peer does not need to write RWI data if such peers are only
Michael Peter Christen [Fri, 8 Feb 2013 16:58:54 +0000 (17:58 +0100)]
A robinson peer does not need to write RWI data if such peers are only
searched using the solr interface. Searching public rpbinsons will be
done with solr only in the future.

5 years agofix for xml blacklist import
Michael Peter Christen [Fri, 8 Feb 2013 14:12:10 +0000 (15:12 +0100)]
fix for xml blacklist import

5 years agofixed a problem with re-feeding of already indexed documents whith
Michael Peter Christen [Fri, 8 Feb 2013 11:45:54 +0000 (12:45 +0100)]
fixed a problem with re-feeding of already indexed documents whith
coordinates attached.

5 years agoAfter the observation that Windows user simply forget that they started
Michael Peter Christen [Thu, 7 Feb 2013 22:39:00 +0000 (23:39 +0100)]
After the observation that Windows user simply forget that they started
YaCy; YaCy is still running and the user additionally expect that
another doubleclick on the YaCy icon simply opens the search windows
(again) I decided to add a function that complies to the expectation to
the user: simply open the browser pop-up page again if the user starts
YaCy while YaCy is still running.

5 years agoMerge branch 'master' of git@gitorious.org:yacy/rc1.git
Marc Nause [Tue, 5 Feb 2013 20:09:41 +0000 (21:09 +0100)]
Merge branch 'master' of git@gitorious.org:yacy/rc1.git