DepSpid

Old News

Sun, 25 May 2008 --- Availability of tasks
As you may already have noticed, there either wasn't much work available during the last few weeks or the available tasks completed very fast. A few different things came together and finally resulted in this situation...
Link: http://www.bjoernhenke.de/forum/viewtopic.php?t=626

Fri, 21 Mar 2008 --- Project status
During the last weeks, the spider application and the server performed very well and together with the help of the participants the project activity has exceeded the amount of 5.5 million visited domains and 2.5 Terabytes total network traffic within half a month. The new spider application v5.27 performs much better than previous versions and has reduced the average memory usage to a fourth. The next steps will be to update the web code of the project server and start with internal testings of the phase-2 application. To do some preparations for this, the project server will be offline for the next two weeks.
Link: http://www.depspid.net/project_activity.php

Tue, 15 Jan 2008 --- New validator running
The new validator is now running and outstanding credits are being granted step by step. It'll take a few days until all pending credits have been granted. The new validator is already producing tasks for the phase-2 application which is currently being developed. That's why the number of unsent tasks is growing fast and you may see "no work from project" messages even if there seems to be enough work available. The display of unsent tasks will be fixed soon to only show the number of phase-1 tasks available until the phase-2 application is ready.

Mon, 07 Jan 2008 --- New tasks
New tasks are currently being produced and send to the clients. It will take a few days until all clients have work again. Please note that the new validator isn't ready yet. The granting of pending credits is supposed to start sometime next week

Tue, 01 Jan 2008 --- Happy new year!
DepSpid wishes all participants a happy, healthy and successful new year! The final tests last year have completed successfully and normal operation is going to be resumed within a few days.

Tue, 18 Dec 2007 --- 24 hours load test
The current version (v5.24) of the spider application seems to have fixed all common errors. The overall error rate is less than 1 percent now. The last batch of new tasks before the Christmas holidays will be a 24 hours load test. This means that there should be enough work for all clients during this time (maximum 10 simultaneous tasks per host). If this batch runs successful, normal operation is going to be restored on 2nd January 2008.

Sun, 16 Dec 2007 --- Third batch of new tasks successful
The third batch of new tasks completed with a lower error rate than the second batch. Most common errors are fixed now but one common error (-11405) remains. The goal of the next batch is to find the cause for the remaining error. 1000 new tasks are currently being prepared.

Wed, 12 Dec 2007 --- Second batch of new tasks completed
The second batch of new tasks completed with a higher error rate than the previous one. This was due to some changes in the task parameters. The reasons for most of these errors have been found and fixed. A new batch of tasks will be sent out together with another application version in a few hours.

Mon, 10 Dec 2007 --- First new batch successful
The first batch with 150 new tasks was sent out yesterday. The returned results are very promising and everything seems to be working very well. The overall error rate is much lower (less than 3%) than with any previous application version before. A second batch with 500 new tasks will be sent out today together with a further improved spider application.

Sat, 08 Dec 2007 --- The end of the outage is coming soon (more or less)
The current outage was planned to end sometime in October. But due to a hardware failure, the outage had to be extended much further. During the last months, the backend database was restructured and a new application version was being build and tested. This restructuring still hasn't finished. But during the next two weeks, there will be some small batches of tasks to test and enhance the new spider application. Normal operation is expected to start again in early January 2008. Please note that the validator/assimilator currently isn't running and credits for the new tasks won't be granted before January.
Link: http://www.bjoernhenke.de/forum/viewtopic.php?t=571

Sun, 11 Nov 2007 --- Project restart delayed
The restart of the project is currently delayed. There are two reasons for this delay: The main reason is a hardware failure which occurred more than one month ago. It took some weeks to find the cause for the hardware failure: a damaged graphics adapter. The faulty graphics adapter was replaced yesterday but the maintenance of the backend database which was the original reason for the long outage was interrupted due to the hardware failure and still needs to be done. The second reason is that a new spider application is currently being build and tested. This new spider version needs to be ready before the project will be restarted. The current estimation is that there will be another two or three weeks before the project will restart.

Wed, 03 Oct 2007 --- Outage reminder
The project is currently suspended for a longer (4 weeks or more) outage. Please don't expect anything (except the main page and the forums) to work during this month, even if some things may work during some times. There will be another news entry as soon as the outage is over.

Sat, 29 Sep 2007 --- Congratulations to team BOINC Synergy
Congratulations to team BOINC Synergy for being the first team that contributed a total of 1 TeraByte network traffic to this project.
Link: http://www.depspid.net/redirect.php?page=team_display.php?teamid=5

Tue, 11 Sep 2007 --- Longer planned outage
Starting at 24th September, the project server will be offline for at least 4 weeks. During the first 10 days of this outage, the project server will be partly online to accept returning results but there won't be new work available. The exact length of the outage depends on the progress of an outstanding restructuring of the backend database. It won't be shorter than 4 weeks but may even least for 5 or 6 weeks if things go bad.

Sun, 12 Aug 2007 --- Milestones for DepSpid
Read more about the future steps of DepSpid...
Link: http://www.bjoernhenke.de/forum/viewtopic.php?t=548

Sat, 11 Aug 2007 --- Separating the application phases
There are plans to separate the two phases of the spider application in the near future. Please vote and/or post a comment in the forum.
Link: http://www.bjoernhenke.de/forum/viewtopic.php?t=547

Thu, 09 Aug 2007 --- New spider application released
A new version of the spider application (v5.12) has been released today. This version fixes a bug which caused much too high CPU load during phase one in some cases (every task with more than a few minutes of CPU time during the network intensive phase can be considered being unusual). The new version should show a more linear progress, too.

Tue, 24 Jul 2007 --- Availability of new tasks
After the last outage, it took multiple days for the project server to produce enough work to keep up with the demand. During this time, a new (faster) task generator was being build and successfully introduced. But as soon as the majority of results kept coming in, the assimilators stopped keeping up with incoming results and produced a backlog of unvalidated results. That's why the task generation was reconfigured to produce much less tasks for the time being.
Link: http://www.bjoernhenke.de/forum/viewtopic.php?t=535

Sat, 30 Jun 2007 --- Recent technical problems
Two big mistakes in the code of the spider application lead into big trouble yesterday. All running DepSpid tasks had to be canceled either manually or remotely. This was really bad timing shortly before the planned outage. The outage will start on next Monday and last for 7 days. The original idea was to supply all hosts with enough work for the time of the outage but now they will have to spend the time without DepSpid tasks. There may be a handful of tasks send out on Sunday to test the new spider application (v5.10) with some changes required to avoid problems like yesterdays in the future but there will be no regular work before the outage again.
Link: http://www.bjoernhenke.de/forum/viewtopic.php?t=520

Fri, 29 Jun 2007 --- Massive technical problems
The DepSpid project is currently having massive technical problems. Please abort all DepSpid tasks currently running phase one (less than 50 percent progress). More info will be posted soon.

Wed, 27 Jun 2007 --- New spider application released
A new version of the spider application (v5.09) has been released today. This version improves checkpointing during phase one while crawling pages with large amounts of links.

Sun, 24 Jun 2007 --- Pending results
Due to the work at the backend database last week the assimilator/validator was offline most of the time. That's why it currently has trouble to catch up with the incoming results. The backlog is already 3 days long and won't catch up before the next outage. Please note that incoming results are assimilated/validated by lowest ID first. This means that older tasks will be preferred and may be processed shortly after being reported while newer task may stay unprocessed for multiple days. A few changes to speed up the whole assimilation process have been successfully implemented during this weekend but I'm still searching for a solution to handle the increasing workload in the future.

Wed, 20 Jun 2007 --- Backend processes running again
All backend processes are running again. However, the transformation of the biggest database table is still in progress and will take some additional days. That's why the server backend doesn't run full speed yet but the changes made in the last days already show their advantages. If things are going smoothly, there should be enough work available for download at the beginning of next week and the maximum number of simultaneous tasks will be increased again.

Mon, 18 Jun 2007 --- Ongoing changes in the backend database
Due to necessary changes in the backend database, the creation of new tasks and the validation of incoming results will be suspended at least for 24 hours.

Tue, 05 Jun 2007 --- Planned outages
There will be two planned outages in the next time. The first (shorter) outage will start next Friday (8th June) and last for 3 days. The second (longer) outage will start around 30th June and last for 8 to 10 days.

Mon, 04 Jun 2007 --- Network connectivity restored
It seems like the carrier was able to repair the broken network connection. There have been no connection errors during the last days and I hope they won't reoccur.

Sat, 26 May 2007 --- New spider application released
A new version of the spider application (v5.08) has been released today. This version should fix the most common error -11404 and help to find the reasons for other errors. It adds some useful information to the result file and has a few more configuration parameters in the task file. Output file compression has been enabled to lower the bandwidth needed for result uploads. To lower the chance of "resource limit exceeded" errors on Win98/ME the maximum flops have been increased by a factor of 1000 because these systems can't measure the CPU time and instead report wall clock time. Phase 1 of the spider process now runs much slower (and therefore longer) to avoid doing harm to visited domains due to high load.

Fri, 25 May 2007 --- Network status
Today, a technician from my ISP was here to find the reason for the connection problems. He discovered that the problem isn't either on my side nor on the ISP side. The problem is located in the "last mile" between my ISP and me. The carrier of the last mile has already been informed but due to the holiday weekend and a strike of the carriers employees, it may take at least a week until the problem is fixed. But the technician gave me a few good tips on how to reduce the effects of the breakdown which seems to work pretty well.

Mon, 21 May 2007 --- Network connection broken
During the last few weeks the internet connection of the project server was interrupted many times for an unknown reason. Things got worse in the last few days and my ISP doesn't yet have an idea what's causing the problems. For testing purposes network cables needed to be reorganized, the DSL modem was exchanged and the router had to be disabled. Until normal network connectivity can be restored, the project server will stay offline most of the time and there will be no new tasks available. Meanwhile a new version of the spider application is being prepared and tested. If the ISP is able to find the reason for the troubles soon, normal operation should be reestablished next week.

Mon, 30 Apr 2007 --- Account creation open
The account creation for DepSpid is now open for everyone and will stay open at least for a while. All pre-registered accounts have been created in the last weeks and there are now already more than 500 registered users.

Mon, 16 Apr 2007 --- Project server back online
The project server is back online again. As being announced earlier, at least BOINC core client version 5.8.0 is required now to participate in the project. Updating to the latest stable version of BOINC (5.8.16) is recommended.
Link: http://boinc.berkeley.edu/download.php

Mon, 02 Apr 2007 --- Outage and minimum core client version
Starting on friday the 6th April, the project server will be offline for 10 days. After this outage, at least BOINC core client version 5.8.0 will be required to further participate in the project. Therefore, please update to the latest stable version of BOINC (5.8.15).
Link: http://boinc.berkeley.edu/download.php

Sat, 10 Mar 2007 --- Recalculation of credits
During the current rebuild of the back-end database, all credits for returned results have been recalculated. Most users will see higher or at least almost the same credits as before the outage but a few people will see less credits on their accounts.
Link: http://www.bjoernhenke.de/forum/viewtopic.php?t=480

Sun, 04 Feb 2007 --- Unplanned outage
Due to a bad crash of the projects back-end database, the creation of new tasks and the validation of incoming results has to be stopped for a while. Before the database will be rebuild, the planned switch to the new server hardware will be made. This is going to happen as soon as possible and you'll most likely see a few website errors during the switch.
Link: http://www.bjoernhenke.de/forum/viewtopic.php?t=463

Fri, 12 Jan 2007 --- Scheduler changes
When the BOINC client requests new work the server will now use the information reported by the client to determine how many tasks are needed to reach the selected number of simultaneous tasks instead of using the information from its own database. This will eleminate problems caused by "ghost WUs". Please note that for non-cpu-intensive projects like DepSpid newer client versions ask for work only when there is none left at the client side. This means that the client gets the selected number of tasks, processes them and waits until the last of them is finished before requesting new DepSpid work again.

Sun, 07 Jan 2007 --- Project server back online again
After a two week outage (one week shorter than planned) the project server is now back online again. It will take some time to catch up with the incoming results but new work should be available within 24 hours. During the outage some hardware was moved around in order to build a new project server. This server will replace the current one in a few months.

Mon, 01 Jan 2007 --- Happy new year!
DepSpid wishes all participants a happy, healthy and successful new year!

News is available as an RSS feed RSS Feed

[Return to DepSpid main page]
DepSpid is alpha software. Use at your own risk.
(c) 2008 Bjoern Henke [Contact] and PerlBOINC community.