rooty/fab-manager
1
0
Fork 0
mirror of https://github.com/LaCasemate/fab-manager.git synced 2024-11-29 10:24:20 +01:00
Code Issues Releases Activity

Merge branch 'dev' for release 2.7.2

Browse Source
This commit is contained in:
Sylvain 2018-11-29 16:56:30 +01:00
commit 187de4510a
6 changed files with 158 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
CHANGELOG.md
View File

@ -1,5 +1,11 @@
# Changelog Fab Manager
# v2.7.2 2018 November 29
- Allow running the ElasticSearch upgrade script while being root
- Fix an issue with ES upgrade script, preventing reindexing in some cases
- Improved ES upgrade documentation
# v2.7.1 2018 November 27
- Updated angular.js to 1.6

131
doc/elastic_upgrade.md
View File

@ -1,5 +1,7 @@
# Instructions for upgrading ElasticSearch
## Automatic upgrade
Fab-manager release 2.6.5 has upgraded its dependency to ElasticSearch from version 1.7 to version 5.6 as the previous was unsupported for months.
To keep using fab-manager you need to upgrade your installation with the new version.
We've wrote a script to automate the process while keeping your data integrity, but there's some requirements to understand before running it.
@ -22,6 +24,133 @@ cd /apps/fabmanager
\curl https://raw.githubusercontent.com/LaCasemate/fab-manager/master/scripts/elastic-upgrade.sh | bash
```
## Manual upgrade
For instructions regarding a manual upgrade, please refer to the official documentation:
- https://www.elastic.co/guide/en/elasticsearch/reference/2.4/restart-upgrade.html
- https://www.elastic.co/guide/en/elasticsearch/reference/5.6/restart-upgrade.html
- https://www.elastic.co/guide/en/elasticsearch/reference/5.6/restart-upgrade.html
## Revert the upgrade
So something goes wrong and the upgrade was not successful.
Sad news, but everything isn't lost, follow this procedure to downgrade to ElasticSearch 1.7.
First, check the status of your indices:
```bash
# Replace fabmanager_elasticsearch_1 in the next command with your container's name.
# You can get it running `docker ps`
ES_IP=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' fabmanager_elasticsearch_1)
curl "$ES_IP:9200/_cat/indices?v"
```
You should get something like this:
```
health status index pri rep docs.count docs.deleted store.size pri.store.size
green open fablab_24 1 0 1944 0 1mb 1mb
green open stats_24 1 0 0 0 2.8mb 104b
green open stats 5 0 13515 0 2.7mb 2.7mb
green open fablab 5 0 1944 4 1.2mb 1.2mb
```
Here, we can see that the migration is not complete, as *docs.count* are not equal for `stat_24` and `stats`.
Now let's stop and remove the ElasticSearch container.
```bash
docker-compose stop elasticsearch
docker-compose rm -f elasticsearch
docker-compose down
```
Then, edit your [docker-compose.yml](../docker/docker-compose.yml) and change the *elasticsearch* block according to the following:
<table>
<tr><td>
<pre style="max-width:350px; overflow-y: scroll">
elasticsearch:
image: elasticsearch:2.4
ulimits:
memlock:
soft: -1
hard: -1
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
volumes:
- ${PWD}/elasticsearch/config:/usr/share/elasticsearch/config
- ${PWD}/elasticsearch:/usr/share/elasticsearch/data
restart: always
</pre>
</td>
<td>
=>
</td>
<td>
<pre style="max-width:350px; overflow-y: scroll">
elasticsearch:
image: elasticsearch:1.7
volumes:
- ${PWD}/elasticsearch:/usr/share/elasticsearch/data
restart: always
</pre>
</td></tr>
</table>
Restart your containers:
```bash
docker-compose pull
docker-compose up -d
```
Now you're back with ElasticSearch 1.7. You can safely restart the upgrade script.
## Debugging the upgrade
You can check for any error during container startup, using:
```bash
docker-compose logs elasticsearch
```
## Skip the upgrade
The upgrade is not working and you can't debug it?
You can skip it by deleting your ES database, installing ES 5.6 and resynchronizing ES from your PostgreSQL database.
**This is not recommended on old installations as this can lead to data losses.**
Here's the procedure:
```bash
curl -XDELETE "$ES_IP:9200/fablab"
curl -XDELETE "$ES_IP:9200/stats"
# delete any other index, if applicable
```
Stop and remove your container, then edit your [docker-compose.yml](../docker/docker-compose.yml) and set the following:
```yml
elasticsearch:
image: elasticsearch:5.6
ulimits:
memlock:
soft: -1
hard: -1
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
volumes:
- ${PWD}/elasticsearch/config:/usr/share/elasticsearch/config
- ${PWD}/elasticsearch:/usr/share/elasticsearch/data
restart: always
```
Copy [elasticsearch.yml](../docker/elasticsearch.yml) and [log4j2.properties](../docker/log4j2.properties) to `elasticsearch/config`, then pull and restart your containers.
Finally reindex your data:
```bash
rake fablab:es_build_stats
rake fablab:generate_stats[3000]
rake fablab:es_build_projects_index
```

3
docker/elasticsearch.yml
View File

@ -1,6 +1,5 @@
http.host: 0.0.0.0
script.inline: true
script.stored: true
script.painless.regex.enabled: true
script.engine.groovy.inline.update: true
path.repo: /usr/share/elasticsearch

6
lib/tasks/fablab.rake
View File

@ -43,10 +43,10 @@ namespace :fablab do
desc '(re)Build ElasticSearch fablab base for stats'
task es_build_stats: :environment do
puts "DELETE stats"
puts 'DELETE stats'
`curl -XDELETE http://#{ENV["ELASTICSEARCH_HOST"]}:9200/stats`
puts "PUT index stats"
puts 'PUT index stats'
`curl -XPUT http://#{ENV["ELASTICSEARCH_HOST"]}:9200/stats -d'
{
"settings" : {
@ -60,7 +60,7 @@ namespace :fablab do
%w[account event machine project subscription training user space].each do |stat|
puts "PUT Mapping stats/#{stat}"
`curl -XPUT http://#{ENV["ELASTICSEARCH_HOST"]}:9200/stats/#{stat}/_mapping -d '
`curl -XPUT http://#{ENV["ELASTICSEARCH_HOST"]}:9200/stats/#{stat}/_mapping -d '
{
"properties": {
"type": {

2
package.json
View File

@ -1,6 +1,6 @@
{
"name": "fab-manager",
"version": "2.7.1",
"version": "2.7.2",
"description": "FabManager is the FabLab management solution. It is web-based, open-source and totally free.",
"keywords": [
"fablab",

18
scripts/elastic-upgrade.sh
View File

@ -308,6 +308,19 @@ upgrade_compose()
then
sed -i.bak "/image: $image/s/.*/&\n ulimits:\n memlock:\n soft: -1\n hard: -1/" "$FM_PATH/docker-compose.yml"
fi
if [ $target = '2.4' ]
then
# get current data directory
dir=$(awk 'BEGIN { FS="\n"; RS="";} { match($0, /image: elasticsearch:2\.4(\n|.)+volumes:(\n|.)+(-.*elasticsearch\/data)/, lines); FS="[ :]+"; RS="\r\n"; split(lines[3], line); print line[2] }' "$FM_PATH/docker-compose.yml")
# set the configuration directory
dir=$(echo "${dir//[$'\t\r\n ']}/config")
# insert configuration directory into docker-compose bindings
awk "BEGIN { FS=\"\n\"; RS=\"\";} { print gensub(/(image: elasticsearch:2\.4(\n|.)+)volumes:\n/, \"\\1volumes:\n - ${dir}:/usr/share/elasticsearch/config\n\", \"g\") }" "$FM_PATH/docker-compose.yml" > "$FM_PATH/.awktmpfile" && mv "$FM_PATH/.awktmpfile" "$FM_PATH/docker-compose.yml"
echo -e "\nCopying ElasticSearch 2.4 configuration files from $(pwd)/docker to $dir..."
mkdir -p "$dir"
curl -sSL https://raw.githubusercontent.com/LaCasemate/fab-manager/master/docker/elasticsearch.yml > "$dir/elasticsearch.yml"
curl -sSL https://raw.githubusercontent.com/LaCasemate/fab-manager/master/docker/log4j2.properties > "$dir/log4j2.properties"
fi
docker-compose pull
docker-compose up -d
wait_for_online
@ -351,7 +364,7 @@ upgrade_docker()
then
image_name="elasticsearch-oss:$target"
image="docker.elastic.co/elasticsearch/$image_name"
elif [ $target = '5.6' ]
elif [ $target = '2.4' ]
then
configdir=$(echo "$volumes" | grep config | awk -F'[ :]' '{print $2}')
echo -e "\nCopying ElasticSearch 2.4 configuration files from $(pwd)/docker to $configdir..."
@ -497,6 +510,9 @@ reindex_indices()
},
"dest": {
"index": "'"$migration_index"'"
},
"script": {
"inline": "ctx._source.remove('"'"'_id'"'"')"
}
}'
done